19

Is there a way to detect the language of the data being entered via the input field?

2
  • 1
    I assume you meant Arabic script, rather than Arabic language? There's a big distinction. Commented Aug 22, 2010 at 12:29
  • 3
    what if the string contains words from multiple languages? example: 私notعرب Commented Aug 22, 2010 at 12:36

10 Answers 10

37

hmm i may offer an improved version of DimaKrasun's function:

functoin is_arabic($string) {
    if($string === 'arabic') {
         return true;
    }
    return false;
}

okay, enough joking!

Pekkas suggestion to use the google translate api is a good one! but you are relying on an external service which is always more complicated etc.

i think Rushyos approch is good! its just not that easy. i wrote the following function for you but its not tested, but it should work...

    <?
function uniord($u) {
    // i just copied this function fron the php.net comments, but it should work fine!
    $k = mb_convert_encoding($u, 'UCS-2LE', 'UTF-8');
    $k1 = ord(substr($k, 0, 1));
    $k2 = ord(substr($k, 1, 1));
    return $k2 * 256 + $k1;
}
function is_arabic($str) {
    if(mb_detect_encoding($str) !== 'UTF-8') {
        $str = mb_convert_encoding($str,mb_detect_encoding($str),'UTF-8');
    }

    /*
    $str = str_split($str); <- this function is not mb safe, it splits by bytes, not characters. we cannot use it
    $str = preg_split('//u',$str); <- this function woulrd probably work fine but there was a bug reported in some php version so it pslits by bytes and not chars as well
    */
    preg_match_all('/.|\n/u', $str, $matches);
    $chars = $matches[0];
    $arabic_count = 0;
    $latin_count = 0;
    $total_count = 0;
    foreach($chars as $char) {
        //$pos = ord($char); we cant use that, its not binary safe 
        $pos = uniord($char);
        echo $char ." --> ".$pos.PHP_EOL;

        if($pos >= 1536 && $pos <= 1791) {
            $arabic_count++;
        } else if($pos > 123 && $pos < 123) {
            $latin_count++;
        }
        $total_count++;
    }
    if(($arabic_count/$total_count) > 0.6) {
        // 60% arabic chars, its probably arabic
        return true;
    }
    return false;
}
$arabic = is_arabic('عربية إخبارية تعمل على مدار اليوم. يمكنك مشاهدة بث القناة من خلال الموقع'); 
var_dump($arabic);
?>

final thoughts: as you see i added for example a latin counter, the range is just a dummy number b ut this way you could detect charsets (hebrew, latin, arabic, hindi, chinese, etc...)

you may also want to eliminate some chars first... maybe @, space, line breaks, slashes etc... the PREG_SPLIT_NO_EMPTY flag for the preg_split function would be useful but because of the bug I didn't use it here.

you can as well have a counter for all the character sets and see which one of course the most...

and finally you should consider chopping your string off after 200 chars or something. this should be enough to tell what character set is used.

and you have to do some error handling! like division by zero, empty string etc etc! don't forget that please... any questions? comment!

if you want to detect the LANGUAGE of a string, you should split into words and check for the words in some pre-defined tables. you don't need a complete dictionary, just the most common words and it should work fine. tokenization/normalization is a must as well! there are libraries for that anyway and this is not what you asked for :) just wanted to mention it

Sign up to request clarification or add additional context in comments.

12 Comments

Your function is making my head go all fuzzy. I'll try to implement it when i'm in a better mood, and let you know if it worked on it. But from what I read, it looks promising.
roger that, don't forget to include the external uniord function on the top! lemme know if ya need any halp
The dictionary is a very good idea, only problem is that outside Latin script you quickly encounter circumstances where external context changes characters - such as multi-glyph characters. You'd have to be careful to avoid context-sensitive characters in your dictionary.
@Rushyo ... what? ... if you split the text into words by whitespaces, tokenice, lower case it and see what you hit in your database. if you hit it, see what relations there are. one word can be in more than one languages. from the hit ratio you should brette easy be able to tell. example: "i am your grandfathers computer xcT4" -> tokenzied "i am your grandfather computer xcT4" assume i, am, your, grandfather are engish words and computer is as well english as german. xcT4 is unknown. you will get 4 vs. 1, good ratio to guess its english
Unicode character properties are your friend... See DimaKrasun's solution. You could still use the 60% rule with a one-liner (which I leave as an exercise to the reader). This answer is unnecessarily complicated (and has poor performance).
|
14

Use regular expression for shorter and easy answer

 $is_arabic = preg_match('/\p{Arabic}/u', $text);

This will return true (1) for arabic string and 0 for non arabic string

1 Comment

This works for me clean and easy. To clarify it more it checks if any part of the string contains Arabic characters. So if its part Arabic part other language it will still return true.
13

this will check if the string is Arabic Or has Arabic text

text must be UNICODE e.g UTF-8

$str = "بسم الله";
if (preg_match('/[اأإء-ي]/ui', $str)) {
    echo "A match was found.";
} else {
    echo "A match was not found.";
}

Comments

3

I assume that in 99% of cases, it is enough to check that string contains Arabic letters and does not consist of all of them.

My core assumption is that if it contains at least two or three Arabic letters, the reader should know how to read it.

You can use a simple function:

<?php
/**
 * Return`s true if string contains only arabic letters.
 *
 * @param string $string
 * @return bool
 */
function contains_arabic($string)
{
    return (preg_match("/^\p{Arabic}/i", $string) > 0);
}

Or if the regex classes do not work:

function contains_arabic($subject)
{
    return (preg_match("/^[\x0600-\x06FF]/i", $subject) > 0);
}

5 Comments

"Is Arabic" != "Contains 'Arabic'" - the question title may be a bit vague, but the question body is more than clear, no?
If string is arabic, it contains arabic letters or not?
Piskvor, DimaKrasun's RegEx ought to indeed detect Arabic characters... not just the string 'Arabic'.
Only reason I proposed my alternative is for speed. RegEx isn't necessarily speedy.
DimaKrasun: Not tested your code, but you appear to have given two different variables names where you intended to have one ($string != $subject)
2

I would use regular expressions to get the number of Arabic characters and compare it to the total length of the string. If the text is for instance at least 60% Arabic charactes, I would consider it as mainly Arabic and apply RTL formatting.

/**
 * Is the given text mainly Arabic language? 
 *
 * @param string $text string to be tested if it is arabic. :-)
 * @return bool 
 */
function ct_is_arabic_text($text) {
    $text = preg_replace('/[ 0-9\(\)\.\,\-\:\n\r_]/', '', $text); // Remove spaces, numbers, punctuation.
    $total_count = mb_strlen($text); // Length of text
    if ($total_count==0)
        return false;
    $arabic_count = preg_match_all("/[اأإء-ي]/ui", $text, $matches); // Number of Arabic characters
    if(($arabic_count/$total_count) > 0.6) { // >60% Arabic chars, its probably Arabic languages
        return true;
    }
    return false;
}

For inline RTL formatting, use CSS. Example class:

.embed-rtl {
 direction: rtl;
 unicode-bidi: normal;
 text-align: right;
}

1 Comment

Th\a\t f\ir\st pa\tt\er\n ha\s \to\o \mu\ch \un\ne\ces\sa\ry e\sc\a\pi\ng.
1

I'm not aware of a PHP solution for this, no.

The Google Translate Ajax APIs may be for you, though.

Check out this Javascript snippet from the API docs: Example: Language Detection

14 Comments

Script detection is a very different thing from language detection.
@Rushyo well, at the moment, he is asking for language detection rather than script.
Taken literally, yes, but I doubt that's the intent.
@Rushyo you don't really know that. I can think of a number of legitimate reasons to try and detect language
In which case, we'd need to know the dialect as well - info not provided.
|
1

I assume you're referring to a Unicode string... in which case, just look for the presence of any character with a code between U+0600–U+06FF (1536–1791) in the string.

2 Comments

the first thing I thought of regex with U+0600–U+06FF, but next was to use \p{Arabic} - in regex, i think \p{Arabic} is the same with U+0600–U+06FF, but i haven`t tried it
I'm pretty sure it's the same, but this method's far quicker.
1
public static function isArabic($string){
    if(preg_match('/\p{Arabic}/u', $string))
        return true;
    return false;
}

Comments

1

The PHP Text_LanguageDetect library is able to detect 52 languages. It's unit-tested and installable via composer and PEAR.

Comments

0

This function checks whether the entered line/sentence is arabic or not. I trimmed it first then check word by word calculating the total count for both.

function isArabic($string){
        // Initializing count variables with zero
        $arabicCount = 0;
        $englishCount = 0;
        // Getting the cleanest String without any number or Brackets or Hyphen
        $noNumbers = preg_replace('/[0-9]+/', '', $string);
        $noBracketsHyphen = array('(', ')', '-');
        $clean = trim(str_replace($noBracketsHyphen , '', $noNumbers));
        // After Getting the clean string, splitting it by space to get the total entered words 
        $array = explode(" ", $clean); // $array contain the words that was entered by the user
        for ($i=0; $i <= count($array) ; $i++) {
            // Checking either word is Arabic or not
            $checkLang = preg_match('/\p{Arabic}/u', $array[$i]);
            if($checkLang == 1){
                ++$arabicCount;
            } else{
                ++$englishCount;
            }
        }
        if($arabicCount >= $englishCount){
            // Return 1 means TRUE i-e Arabic
            return 1;
        } else{
            // Return 0 means FALSE i-e English
            return 0;
        }
    }

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.