Unicode::Normalize - Unicode Normalization Forms |
Unicode::Normalize - Unicode Normalization Forms
use Unicode::Normalize;
$NFD_string = NFD($string); # Normalization Form D $NFC_string = NFC($string); # Normalization Form C $NFKD_string = NFKD($string); # Normalization Form KD $NFKC_string = NFKC($string); # Normalization Form KC
or
use Unicode::Normalize 'normalize';
$NFD_string = normalize('D', $string); # Normalization Form D $NFC_string = normalize('C', $string); # Normalization Form C $NFKD_string = normalize('KD', $string); # Normalization Form KD $NFKC_string = normalize('KC', $string); # Normalization Form KC
$NFD_string = NFD($string)
$NFC_string = NFC($string)
$NFKD_string = NFKD($string)
$NFKC_string = NFKC($string)
$normalized_string = normalize($form_name, $string)
$form_name
, one of the following names must be given.
'C' or 'NFC' for Normalization Form C 'D' or 'NFD' for Normalization Form D 'KC' or 'NFKC' for Normalization Form KC 'KD' or 'NFKD' for Normalization Form KD
$decomposed_string = decompose($string)
$decomposed_string = decompose($string, $useCompatMapping)
If the second parameter (a boolean) is omitted or false, decomposes it using the Canonical Decomposition Mapping. If true, decomposes it using the Compatibility Decomposition Mapping.
The string returned is not always in NFD/NFKD. Reordering may be required.
$NFD_string = reorder(decompose($string)); # eq. to NFD() $NFKD_string = reorder(decompose($string, TRUE)); # eq. to NFKD()
$reordered_string = reorder($string)
E.g., when you have a list of NFD/NFKD strings, you can get the concatenated NFD/NFKD string from them, saying
$concat_NFD = reorder(join '', @NFD_strings); $concat_NFKD = reorder(join '', @NFKD_strings);
$composed_string = compose($string)
E.g., when you have a NFD/NFKD string, you can get its NFC/NFKC string, saying
$NFC_string = compose($NFD_string); $NFKC_string = compose($NFKD_string);
(see Annex 8, UAX #15; DerivedNormalizationProps.txt)
The following functions check whether the string is in that normalization form.
The result returned will be:
YES The string is in that normalization form. NO The string is not in that normalization form. MAYBE Dubious. Maybe yes, maybe no.
$result = checkNFD($string)
YES
(1
) or NO
(empty string
).
$result = checkNFC($string)
YES
(1
), NO
(empty string
), or MAYBE
(undef
).
$result = checkNFKD($string)
YES
(1
) or NO
(empty string
).
$result = checkNFKC($string)
YES
(1
), NO
(empty string
), or MAYBE
(undef
).
$result = check($form_name, $string)
YES
(1
), NO
(empty string
), or MAYBE
(undef
).
$form_name
is alike to that for normalize()
.
Note
In the cases of NFD and NFKD, the answer must be either YES
or NO
.
The answer MAYBE
may be returned in the cases of NFC and NFKC.
A MAYBE-NFC/NFKC string should contain at least
one combining character or the like.
For example, COMBINING ACUTE ACCENT
has
the MAYBE_NFC/MAYBE_NFKC property.
Both checkNFC("A\N{COMBINING ACUTE ACCENT}")
and checkNFC("B\N{COMBINING ACUTE ACCENT}")
will return MAYBE
.
"A\N{COMBINING ACUTE ACCENT}"
is not in NFC
(its NFC is "\N{LATIN CAPITAL LETTER A WITH ACUTE}"
),
while "B\N{COMBINING ACUTE ACCENT}"
is in NFC.
If you want to check exactly, compare the string with its NFC/NFKC; i.e.,
$string eq NFC($string) # more thorough than checkNFC($string) $string eq NFKC($string) # more thorough than checkNFKC($string)
These functions are interface of character data used internally. If you want only to get Unicode normalization forms, you don't need call them yourself.
$canonical_decomposed = getCanon($codepoint)
If it is not decomposable, returns undef
.
$compatibility_decomposed = getCompat($codepoint)
If it is not decomposable, returns undef
.
$codepoint_composite = getComposite($codepoint_here, $codepoint_next)
If they are not composable, returns undef
.
$combining_class = getCombinClass($codepoint)
$is_exclusion = isExclusion($codepoint)
$is_singleton = isSingleton($codepoint)
$is_non_startar_decomposition = isNonStDecomp($codepoint)
$may_be_composed_with_prev_char = isComp2nd($codepoint)
NFC
, NFD
, NFKC
, NFKD
: by default.
normalize
and other some functions: on request.
SADAHIRO Tomoyuki, <SADAHIRO@cpan.org>
http://homepage1.nifty.com/nomenclator/perl/
Copyright(C) 2001-2002, SADAHIRO Tomoyuki. Japan. All rights reserved.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
Unicode::Normalize - Unicode Normalization Forms |