General Encoding Principles Encoding Special Characters in Web Output

and t hus you m ust be prepared t o encode it . Neglect ing t o do t his oft en result s in script s t hat generat e pages cont aining m alform ed HTML t hat displays incorrect ly. This sect ion describes how t o handle special charact ers, beginning wit h som e general principles, and t hen discusses how each API im plem ent s encoding support . The API - specific exam ples show how t o process inform at ion drawn from a dat abase t able, but t hey can be adapt ed t o any cont ent you include in a web page, no m at t er it s source.

16.5.4 General Encoding Principles

One form of encoding applies t o charact ers t hat are used in writ ing HTML const ruct s, anot her applies t o t ext t hat is included in URLs. I t s im port ant t o underst and t his dist inct ion so t hat you dont encode t ext inappropriat ely. Not e t oo t hat encoding t ext for inclusion in a web page is an ent irely different issue t han encoding special charact ers in dat a values for inclusion in a SQL st at em ent . The lat t er issue is discussed in Recipe 2.8 .

16.5.4.1 Encoding characters that are special in HTML

HTML m arkup uses and charact ers t o begin and end t ags, t o begin special ent it y nam es such as nbsp; t o signify a non- breaking space , and t o quot e at t ribut e values in t ags such as p align=left . Consequent ly, t o display lit eral inst ances of t hese charact ers, you m ust encode t hem as HTML ent it ies so t hat browsers or ot her client s underst and your int ent . To do t his, convert , , , and t o t he corresponding HTML ent it y designat ors lt; less t han , gt; great er t han , amp; am persand , and quot; quot e . Suppose you want t o display t he following st ring lit erally in a web page: Paragraphs begin and end with p p tags. I f you send t his t ext t o t he client browser exact ly as shown, t he browser will m isint erpret it . The p and p t ags w ill be t aken as paragraph m arkers and t he m ay be t aken as t he beginning of an HTML ent it y designat or. To display t he st ring t he way you int end, t he special charact ers should be encoded as t he lt; , gt; , and amp; , ent it ies: Paragraphs begin and end with lt;pgt; amp; lt;pgt; tags. The principle of encoding t ext t his way is also useful wit hin t ags. For exam ple, HTML t ag at t ribut e values usually are enclosed wit hin double quot es, so it s im port ant t o perform HTML- encoding on at t ribut e values. Suppose you want t o include a t ext input box in a form , and you want t o provide an init ial value of Rich Goose Gossage t o be displayed in t he box. You cannot writ e t hat value lit erally in t he t ag like t his: input type=text name=player_name value=Rich Goose Gossage The problem here is t hat t he double-quot ed value at t ribut e includes int ernal double quot es, which m akes t he input t ag m alform ed. The proper way t o writ e it is t o encode t he double quot es: input type=text name=player_name value=Rich quot;Goosequot; Gossage When a browser receives t his t ext , it will decode t he quot; ent it ies back t o charact ers and int erpret t he value at t ribut e value properly.

16.5.4.2 Encoding characters that are special in URLs

URLs for hyperlinks t hat occur wit hin HTML pages have t heir own synt ax, and t heir own encoding. This encoding applies t o at t ribut es wit hin several t ags: a href= URL img src= URL form action= URL frame src= URL Many charact ers have special m eaning wit hin URLs, such as : , , ? , = , , and ; . The following URL cont ains som e of t hese charact ers: ht t p: apache.snake.net m yscript .php?id= 428nam e= Gandalf Here t he : and charact ers segm ent t he URL int o com ponent s, t he ? charact er indicat es t hat param et ers are present , and t he charact ers separat es t he param et ers, each of which is specified as a name = value pair . The ; charact er is not present in t he URL j ust shown, but com m only is used inst ead of t o separat e param et ers. I f you want t o include any of t hese charact ers lit erally wit hin a URL, you m ust encode t hem t o prevent t he browser from int erpret ing t hem wit h t heir usual special m eaning. Ot her charact ers such as spaces require special t reat m ent as well. Spaces are not allowed wit hin a URL, so if you want t o reference a page nam ed m y hom e page.ht m l on t he sit e apache.snake.net , t he URL in t he following hyperlink wont work: a href=http:apache.snake.netmy home page.htmlMy Home Pagea URL- encoding for special and reserved charact ers is perform ed by convert ing each such charact er t o followed by t wo hexadecim al digit s represent ing t he charact ers ASCI I code. For exam ple, t he ASCI I value of t he space charact er is 32 decim al, or 20 hexadecim al, so youd writ e t he preceding hyperlink like t his: Mya href=http:apache.snake.netmy20home20page.htmlMy Home Pagea Som et im es youll see spaces encoded as + in URLs. This t oo is legal.

16.5.4.3 Encoding interactions

Be sure t o encode inform at ion properly for t he cont ext in which youre using it . Suppose you want t o creat e a hyperlink t o t rigger a search for it em s m at ching a search t erm , and you want t he t erm it self t o appear as t he link label t hat is displayed in t he page. I n t his case, t he t erm appears as a param et er in t he URL, and also as HTML t ext bet ween t he a and a t ags. I f t he search t erm is cat s dogs , t he unencoded hyperlink const ruct looks like t his: a href=cgi-binmyscript?term=cats dogscats dogsa That is incorrect because is special in bot h cont ext s and t he spaces are special in t he URL. The link should be writ t en like t his inst ead: a href=cgi-binmyscript?term=cats202620dogscats amp; dogsa Here, is HTML-encoded as amp; for t he link label, and is URL-encoded as 26 for t he URL, which also includes spaces encoded as 20 . Grant ed, it s a pain t o encode t ext before writ ing it t o a web page, and som et im es you know enough about a value t hat you can skip t he encoding see t he sidebar Do You Always Need t o Encode Web Page Out put ? . But encoding is t he safe t hing t o do m ost of t he t im e. Fort unat ely, m ost API s provide funct ions t o do t he work for you. This m eans you need not know every charact er t hat is special in a given cont ext . You j ust need t o know which kind of encoding t o perform , and call t he appropriat e funct ion t o produce t he int ended result . Do You Always Need to Encode Web Page Output? I f you know a value is legal in a part icular cont ext wit hin a web page, you need not encode it . For exam ple, if you obt ain a value from an int eger- valued colum n in a dat abase t able t hat cannot be NULL , it m ust necessarily be an int eger. No HTML- or URL- encoding is needed t o include t he value in a web page, because digit s are not special in HTML t ext or wit hin URLs. On t he ot her hand, suppose you solicit an int eger value using a field in a web form . You m ight be expect ing t he user t o provide an int eger, but t he user m ight be confused and ent er an illegal value. You could handle t his by displaying an error page t hat shows t he value and explains t hat it s not an int eger. But if t he value cont ains special charact ers and you dont encode it , t he page w ont display t he value properly, possibly confusing t he user furt her.

16.5.5 Encoding Special Characters Using Web APIs