How do I check if the charset of a string is UTF8?
Add a comment
|
7 Answers
Don't reinvent the wheel. There is a builtin function for that task: mb_check_encoding().
mb_check_encoding($string, 'UTF-8');
2 Comments
CMCDragonkai
What happens if you don't have that extension installed?
soulmerge
@CMCDragonkai If your data is in a multibyte format (like UTF-8), you should definitely install the PHP extension that provides the functions to correctly operate on your input. If you use
strlen on an UTF-8 string, for example, you will get the wrong length if the input contains non-ASCII characters.Just a side note:
You cannot determine if a given string is encoded in UTF-8. You only can determine if a given string is definitively not encoded in UTF-8. Please see a related question here:
You cannot detect if a given string (or byte sequence) is a UTF-8 encoded text as for example each and every series of UTF-8 octets is also a valid (if nonsensical) series of Latin-1 (or some other encoding) octets. However not every series of valid Latin-1 octets are valid UTF-8 series.
Comments
function is_utf8($string) {
return preg_match('%^(?:
[\x09\x0A\x0D\x20-\x7E] # ASCII
| [\xC2-\xDF][\x80-\xBF] # non-overlong 2-byte
| \xE0[\xA0-\xBF][\x80-\xBF] # excluding overlongs
| [\xE1-\xEC\xEE\xEF][\x80-\xBF]{2} # straight 3-byte
| \xED[\x80-\x9F][\x80-\xBF] # excluding surrogates
| \xF0[\x90-\xBF][\x80-\xBF]{2} # planes 1-3
| [\xF1-\xF3][\x80-\xBF]{3} # planes 4-15
| \xF4[\x80-\x8F][\x80-\xBF]{2} # plane 16
)*$%xs', $string);
}
I have checked. This function is effective.
Comments
Better yet, use both of the above solutions.
function isUtf8($string) {
if (function_exists("mb_check_encoding") && is_callable("mb_check_encoding")) {
return mb_check_encoding($string, 'UTF8');
}
return preg_match('%^(?:
[\x09\x0A\x0D\x20-\x7E] # ASCII
| [\xC2-\xDF][\x80-\xBF] # non-overlong 2-byte
| \xE0[\xA0-\xBF][\x80-\xBF] # excluding overlongs
| [\xE1-\xEC\xEE\xEF][\x80-\xBF]{2} # straight 3-byte
| \xED[\x80-\x9F][\x80-\xBF] # excluding surrogates
| \xF0[\x90-\xBF][\x80-\xBF]{2} # planes 1-3
| [\xF1-\xF3][\x80-\xBF]{3} # planes 4-15
| \xF4[\x80-\x8F][\x80-\xBF]{2} # plane 16
)*$%xs', $string);
}
3 Comments
soulmerge
Call to both function_exists() and is_callable() is redundant, you can skip one of them. But since is_callable() is intended to check if the parameter is a valid callback (see link below), I would use function_exists() to make it more readable. Link to callback pseudo-type: php.net/manual/en/…
JDelage
@nikc.org - I don't understand - why do you want to test whether the function
mb_check_encoding exists?
nikc.org
@JDelage because it's part of an extension (php.net/manual/en/book.mbstring.php) that is not guaranteed to always be present.
mb_detect_encoding($string); will return the actual character set of $string. mb_check_encoding($string, 'UTF-8'); will return TRUE if character set of $string is UTF-8 else FALSE
Comments
None of the above answers are correct. Yes, they may be working. If you take the answer with the preg_replace function, are you trying to kill your server if you process a lot of stirng ? Use this pure PHP function with no regex, work 100% of the time and it's way faster.
if(function_exists('grk_Is_UTF8') === FALSE){
function grk_Is_UTF8($String=''){
# On va calculer la longeur de la chaîne
$Len = strlen($String);
# On va boucler sur chaque caractère
for($i = 0; $i < $Len; $i++){
# On va aller chercher la valeur ASCII du caractère
$Ord = ord($String[$i]);
if($Ord > 128){
if($Ord > 247){
return FALSE;
} elseif($Ord > 239){
$Bytes = 4;
} elseif($Ord > 223){
$Bytes = 3;
} elseif($Ord > 191){
$Bytes = 2;
} else {
return FALSE;
}
#
if(($i + $Bytes) > $Len){
return FALSE;
}
# On va boucler sur chaque bytes / caractères
while($Bytes > 1){
# +1
$i++;
# On va aller chercher la valeur ASCII du caractère / byte
$Ord = ord($String[$i]);
if($Ord < 128 OR $Ord > 191){
return FALSE;
}
# Parfait
$Bytes--;
}
}
}
# Vrai
return TRUE;
}
}
