1 <?xml version="1.0" encoding="ISO-8859-1"?>
3 PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
4 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
6 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
8 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
9 <meta name="AUTHOR" content="pme@gcc.gnu.org (Phil Edwards)" />
10 <meta name="KEYWORDS" content="HOWTO, libstdc++, GCC, g++, libg++, STL" />
11 <meta name="DESCRIPTION" content="HOWTO for the libstdc++ chapter 21." />
12 <meta name="GENERATOR" content="vi and eight fingers" />
13 <title>libstdc++-v3 HOWTO: Chapter 21</title>
14 <link rel="StyleSheet" href="../lib3styles.css" />
18 <h1 class="centered"><a name="top">Chapter 21: Strings</a></h1>
20 <p>Chapter 21 deals with the C++ strings library (a welcome relief).
24 <!-- ####################################################### -->
28 <li><a href="#1">MFC's CString</a></li>
29 <li><a href="#2">A case-insensitive string class</a></li>
30 <li><a href="#3">Breaking a C++ string into tokens</a></li>
31 <li><a href="#4">Simple transformations</a></li>
32 <li><a href="#5">Making strings of arbitrary character types</a></li>
37 <!-- ####################################################### -->
39 <h2><a name="1">MFC's CString</a></h2>
40 <p>A common lament seen in various newsgroups deals with the Standard
41 string class as opposed to the Microsoft Foundation Class called
42 CString. Often programmers realize that a standard portable
43 answer is better than a proprietary nonportable one, but in porting
44 their application from a Win32 platform, they discover that they
45 are relying on special functions offered by the CString class.
47 <p>Things are not as bad as they seem. In
48 <a href="http://gcc.gnu.org/ml/gcc/1999-04n/msg00236.html">this
49 message</a>, Joe Buck points out a few very important things:
52 <li>The Standard <code>string</code> supports all the operations
53 that CString does, with three exceptions.
55 <li>Two of those exceptions (whitespace trimming and case
56 conversion) are trivial to implement. In fact, we do so
59 <li>The third is <code>CString::Format</code>, which allows formatting
60 in the style of <code>sprintf</code>. This deserves some mention:
63 <p><a name="1.1internal"> <!-- Coming from Chapter 27 -->
64 The old libg++ library had a function called form(), which did much
65 the same thing. But for a Standard solution, you should use the
66 stringstream classes. These are the bridge between the iostream
67 hierarchy and the string class, and they operate with regular
68 streams seamlessly because they inherit from the iostream
69 hierarchy. An quick example:
73 #include <iostream>
74 #include <string>
75 #include <sstream>
77 string f (string& incoming) // incoming is "foo N"
79 istringstream incoming_stream(incoming);
83 incoming_stream >> the_word // extract "foo"
84 >> the_number; // extract N
86 ostringstream output_stream;
87 output_stream << "The word was " << the_word
88 << " and 3*N was " << (3*the_number);
90 return output_stream.str();
92 <p>A serious problem with CString is a design bug in its memory
93 allocation. Specifically, quoting from that same message:
96 CString suffers from a common programming error that results in
97 poor performance. Consider the following code:
99 CString n_copies_of (const CString& foo, unsigned n)
102 for (unsigned i = 0; i < n; i++)
107 This function is O(n^2), not O(n). The reason is that each +=
108 causes a reallocation and copy of the existing string. Microsoft
109 applications are full of this kind of thing (quadratic performance
110 on tasks that can be done in linear time) -- on the other hand,
111 we should be thankful, as it's created such a big market for high-end
114 If you replace CString with string in the above function, the
117 <p>Joe Buck also pointed out some other things to keep in mind when
118 comparing CString and the Standard string class:
121 <li>CString permits access to its internal representation; coders
122 who exploited that may have problems moving to <code>string</code>.
124 <li>Microsoft ships the source to CString (in the files
125 MFC\SRC\Str{core,ex}.cpp), so you could fix the allocation
126 bug and rebuild your MFC libraries.
127 <em><strong>Note:</strong> It looks like the the CString shipped
128 with VC++6.0 has fixed this, although it may in fact have been
129 one of the VC++ SPs that did it.</em>
131 <li><code>string</code> operations like this have O(n) complexity
132 <em>if the implementors do it correctly</em>. The libstdc++
133 implementors did it correctly. Other vendors might not.
135 <li>While parts of the SGI STL are used in libstdc++-v3, their
136 string class is not. The SGI <code>string</code> is essentially
137 <code>vector<char></code> and does not do any reference
138 counting like libstdc++-v3's does. (It is O(n), though.)
139 So if you're thinking about SGI's string or rope classes,
140 you're now looking at four possibilities: CString, the
141 libstdc++ string, the SGI string, and the SGI rope, and this
142 is all before any allocator or traits customizations! (More
143 choices than you can shake a stick at -- want fries with that?)
146 <p>Return <a href="#top">to top of page</a> or
147 <a href="../faq/index.html">to the FAQ</a>.
151 <h2><a name="2">A case-insensitive string class</a></h2>
152 <p>The well-known-and-if-it-isn't-well-known-it-ought-to-be
153 <a href="http://www.peerdirect.com/resources/">Guru of the Week</a>
154 discussions held on Usenet covered this topic in January of 1998.
155 Briefly, the challenge was, "write a 'ci_string' class which
156 is identical to the standard 'string' class, but is
157 case-insensitive in the same way as the (common but nonstandard)
158 C function stricmp():"
161 ci_string s( "AbCdE" );
164 assert( s == "abcde" );
165 assert( s == "ABCDE" );
167 // still case-preserving, of course
168 assert( strcmp( s.c_str(), "AbCdE" ) == 0 );
169 assert( strcmp( s.c_str(), "abcde" ) != 0 ); </pre>
171 <p>The solution is surprisingly easy. The original answer pages
172 on the GotW website were removed into cold storage, in
174 <a href="http://cseng.aw.com/bookpage.taf?ISBN=0-201-61562-2">a
175 published book of GotW notes</a>. Before being
176 put on the web, of course, it was posted on Usenet, and that
177 posting containing the answer is <a href="gotw29a.txt">available
180 <p>See? Told you it was easy!</p>
181 <p><strong>Added June 2000:</strong> The May issue of <u>C++ Report</u>
183 a fascinating article by Matt Austern (yes, <em>the</em> Matt Austern)
184 on why case-insensitive comparisons are not as easy as they seem,
185 and why creating a class is the <em>wrong</em> way to go about it in
186 production code. (The GotW answer mentions one of the principle
187 difficulties; his article mentions more.)
189 <p>Basically, this is "easy" only if you ignore some things,
190 things which may be too important to your program to ignore. (I chose
191 to ignore them when originally writing this entry, and am surprised
192 that nobody ever called me on it...) The GotW question and answer
193 remain useful instructional tools, however.
195 <p><strong>Added September 2000:</strong> James Kanze provided a link to a
196 <a href="http://www.unicode.org/unicode/reports/tr21/">Unicode
197 Technical Report discussing case handling</a>, which provides some
198 very good information.
200 <p>Return <a href="#top">to top of page</a> or
201 <a href="../faq/index.html">to the FAQ</a>.
205 <h2><a name="3">Breaking a C++ string into tokens</a></h2>
206 <p>The Standard C (and C++) function <code>strtok()</code> leaves a lot to
207 be desired in terms of user-friendliness. It's unintuitive, it
208 destroys the character string on which it operates, and it requires
209 you to handle all the memory problems. But it does let the client
210 code decide what to use to break the string into pieces; it allows
211 you to choose the "whitespace," so to speak.
213 <p>A C++ implementation lets us keep the good things and fix those
214 annoyances. The implementation here is more intuitive (you only
215 call it once, not in a loop with varying argument), it does not
216 affect the original string at all, and all the memory allocation
219 <p>It's called stringtok, and it's a template function. It's given
220 <a href="stringtok_h.txt">in this file</a> in a less-portable form than
221 it could be, to keep this example simple (for example, see the
222 comments on what kind of string it will accept). The author uses
223 a more general (but less readable) form of it for parsing command
224 strings and the like. If you compiled and ran this code using it:
227 std::list<string> ls;
228 stringtok (ls, " this \t is\t\n a test ");
229 for (std::list<string>const_iterator i = ls.begin();
232 std::cerr << ':' << (*i) << ":\n";
234 <p>You would see this as output:
241 <p>with all the whitespace removed. The original <code>s</code> is still
242 available for use, <code>ls</code> will clean up after itself, and
243 <code>ls.size()</code> will return how many tokens there were.
245 <p>As always, there is a price paid here, in that stringtok is not
246 as fast as strtok. The other benefits usually outweight that, however.
247 <a href="stringtok_std_h.txt">Another version of stringtok is given
248 here</a>, suggested by Chris King and tweaked by Petr Prikryl,
249 and this one uses the
250 transformation functions mentioned below. If you are comfortable
251 with reading the new function names, this version is recommended
254 <p><strong>Added February 2001:</strong> Mark Wilden pointed out that the
255 standard <code>std::getline()</code> function can be used with standard
256 <a href="../27_io/howto.html">istringstreams</a> to perform
257 tokenizing as well. Build an istringstream from the input text,
258 and then use std::getline with varying delimiters (the three-argument
259 signature) to extract tokens into a string.
261 <p>Return <a href="#top">to top of page</a> or
262 <a href="../faq/index.html">to the FAQ</a>.
266 <h2><a name="4">Simple transformations</a></h2>
267 <p>Here are Standard, simple, and portable ways to perform common
268 transformations on a <code>string</code> instance, such as "convert
269 to all upper case." The word transformations is especially
270 apt, because the standard template function
271 <code>transform<></code> is used.
273 <p>This code will go through some iterations (no pun). Here's the
274 simplistic version usually seen on Usenet:
277 #include <string>
278 #include <algorithm>
279 #include <cctype> // old <ctype.h>
283 char operator() (char c) const { return std::tolower(c); }
288 char operator() (char c) const { return std::toupper(c); }
293 std::string s ("Some Kind Of Initial Input Goes Here");
295 // Change everything into upper case
296 std::transform (s.begin(), s.end(), s.begin(), ToUpper());
298 // Change everything into lower case
299 std::transform (s.begin(), s.end(), s.begin(), ToLower());
301 // Change everything back into upper case, but store the
302 // result in a different string
303 std::string capital_s;
304 capital_s.resize(s.size());
305 std::transform (s.begin(), s.end(), capital_s.begin(), ToUpper());
307 <p><span class="larger"><strong>Note</strong></span> that these calls all
308 involve the global C locale through the use of the C functions
309 <code>toupper/tolower</code>. This is absolutely guaranteed to work --
310 but <em>only</em> if the string contains <em>only</em> characters
311 from the basic source character set, and there are <em>only</em>
312 96 of those. Which means that not even all English text can be
313 represented (certain British spellings, proper names, and so forth).
314 So, if all your input forevermore consists of only those 96
315 characters (hahahahahaha), then you're done.
317 <p><span class="larger"><strong>Note</strong></span> that the
318 <code>ToUpper</code> and <code>ToLower</code> function objects
319 are needed because <code>toupper</code> and <code>tolower</code>
320 are overloaded names (declared in <code><cctype></code> and
321 <code><locale></code>) so the template-arguments for
322 <code>transform<></code> cannot be deduced, as explained in
323 <a href="http://gcc.gnu.org/ml/libstdc++/2002-11/msg00180.html">this
324 message</a>. <!-- section 14.8.2.4 clause 16 in ISO 14882:1998
325 if you're into that sort of thing -->
326 At minimum, you can write short wrappers like
329 char toLower (char c)
331 return std::tolower(c);
333 <p>The correct method is to use a facet for a particular locale
334 and call its conversion functions. These are discussed more in
335 Chapter 22; the specific part is
336 <a href="../22_locale/howto.html#7">Correct Transformations</a>,
337 which shows the final version of this code. (Thanks to James Kanze
338 for assistance and suggestions on all of this.)
340 <p>Another common operation is trimming off excess whitespace. Much
341 like transformations, this task is trivial with the use of string's
342 <code>find</code> family. These examples are broken into multiple
343 statements for readability:
346 std::string str (" \t blah blah blah \n ");
348 // trim leading whitespace
349 string::size_type notwhite = str.find_first_not_of(" \t\n");
350 str.erase(0,notwhite);
352 // trim trailing whitespace
353 notwhite = str.find_last_not_of(" \t\n");
354 str.erase(notwhite+1); </pre>
355 <p>Obviously, the calls to <code>find</code> could be inserted directly
356 into the calls to <code>erase</code>, in case your compiler does not
357 optimize named temporaries out of existence.
359 <p>Return <a href="#top">to top of page</a> or
360 <a href="../faq/index.html">to the FAQ</a>.
364 <h2><a name="5">Making strings of arbitrary character types</a></h2>
365 <p>The <code>std::basic_string</code> is tantalizingly general, in that
366 it is parameterized on the type of the characters which it holds.
367 In theory, you could whip up a Unicode character class and instantiate
368 <code>std::basic_string<my_unicode_char></code>, or assuming
369 that integers are wider than characters on your platform, maybe just
370 declare variables of type <code>std::basic_string<int></code>.
372 <p>That's the theory. Remember however that basic_string has additional
373 type parameters, which take default arguments based on the character
374 type (called CharT here):
377 template <typename CharT,
378 typename Traits = char_traits<CharT>,
379 typename Alloc = allocator<CharT> >
380 class basic_string { .... };</pre>
381 <p>Now, <code>allocator<CharT></code> will probably Do The Right
382 Thing by default, unless you need to implement your own allocator
385 <p>But <code>char_traits</code> takes more work. The char_traits
386 template is <em>declared</em> but not <em>defined</em>.
387 That means there is only
390 template <typename CharT>
393 static void foo (type1 x, type2 y);
396 <p>and functions such as char_traits<CharT>::foo() are not
397 actually defined anywhere for the general case. The C++ standard
398 permits this, because writing such a definition to fit all possible
399 CharT's cannot be done. (For a time, in earlier versions of GCC,
400 there was a mostly-correct implementation that let programmers be
401 lazy. :-) But it broke under many situations, so it was removed.
402 You are no longer allowed to be lazy and non-portable.)
404 <p>The C++ standard also requires that char_traits be specialized for
405 instantiations of <code>char</code> and <code>wchar_t</code>, and it
406 is these template specializations that permit entities like
407 <code>basic_string<char,char_traits<char>></code> to work.
409 <p>If you want to use character types other than char and wchar_t,
410 such as <code>unsigned char</code> and <code>int</code>, you will
411 need to write specializations for them at the present time. If you
412 want to use your own special character class, then you have
413 <a href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00163.html">a lot
414 of work to do</a>, especially if you with to use i18n features
415 (facets require traits information but don't have a traits argument).
417 <p>One example of how to specialize char_traits is given <a
418 href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00260.html">in
419 this message</a>, which was then put into the file <code>
420 include/ext/pod_char_traits.h</code> at a later date. We agree
421 that the way it's used with basic_string (scroll down to main())
422 doesn't look nice, but that's because <a
423 href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00236.html">the
424 nice-looking first attempt</a> turned out to <a
425 href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00242.html">not
426 be conforming C++</a>, due to the rule that CharT must be a POD.
427 (See how tricky this is?)
429 <p>Other approaches were suggested in that same thread, such as providing
430 more specializations and/or some helper types in the library to assist
431 users writing such code. So far nobody has had the time...
432 <a href="../17_intro/contribute.html">do you?</a>
434 <p>Return <a href="#top">to top of page</a> or
435 <a href="../faq/index.html">to the FAQ</a>.
439 <!-- ####################################################### -->
442 <p class="fineprint"><em>
443 See <a href="../17_intro/license.html">license.html</a> for copying conditions.
444 Comments and suggestions are welcome, and may be sent to
445 <a href="mailto:libstdc++@gcc.gnu.org">the libstdc++ mailing list</a>.