Locale-Sensitive C/C++ String Operation Function
char *strtok(char *strToken, const char *strDelimit);
wchar_t *wcstok(wchar_t *strToken, const wchar_t *strDelimit);
unsigned char *_mbstok(unsigned char *strToken, const unsigned char *strDelimit);
_TXCHAR *_tcstok(_TXCHAR *strToken, _TXCHAR *strDelimit);
Internationalization (I18n) Function Overview
The strtok function returns a pointer to the next token found in
the string strToken , and returns NULL when no more tokens are found.
Each call to the function also modifies strToken by substituting a
NULL character for the encountered delimiter, specified by strDelimit .
wcstok is the wide character equivalent; its parameters and return are wide-character strings.
_mbstok is supported on Windows platforms only and is the multibyte equivalent; its parameters and return are multibyte strings.
_tcstok is the Windows-only Generic version of the function; with the
_MBCS or _UNICODE compiler flags determining its mapping to either
_mbstok or wcstok .
I18n Issues
There is no ANSI multibyte UTF-8 version of this function, and strtok does not work correctly with multibyte UTF-8 characters. (In a string containing multibyte UTF-8 characters, characters consisting of more than one byte are not treated by strtok as an entity, each byte is treated separately.)
Therefore, if any function parameters may possibly contain multibyte UTF-8 characters, the parameters will have to be converted to wide characters (wchar_t ) and then the wide function wcstok can be used.
For Windows MBCS platforms, ensure that the multibyte code page is set properly, as _mbstok depends on it. By default, the multibyte code page is set to the system-default ANSI code page obtained from the operating system at program startup. Use _getmbcp and _setmbcp to query or change the current multibyte code page, respectively.
Recommended Replacements*
*If you're already using the recommended function, see I18n Issues for other reasons why Globalyzer is detecting the function.
Locale-Sensitive C/C++
String Operation Functions
|