Smart Quotes for Docutils

Author:: Günter Milde, based on SmartyPants by John Gruber, Brad Choate, and Chad Miller
Contact:: docutils-develop@lists.sourceforge.net
Revision:: 9051
Date:: 2022-04-02
License:: Released under the terms of the 2-Clause BSD license

Abstract

This document describes the Docutils smartquotes module.

Description

The "smart_quotes" configuration setting triggers the SmartQuotes transformation on Text nodes that includes the following steps:

Straight quotes (" and ') into "curly" quote characters
dashes (-- and ---) into en- and em-dash entities
three consecutive dots (... or . . .) into an ellipsis entity.

This means you can write, edit, and save your documents using plain old ASCII -- straight quotes, plain dashes, and plain dots -- while Docutils generates documents with typographical quotes, dashes, and ellipses.

Advantages:

Typing speed (especially when blind-typing).
The possibility to change the quoting style of the complete document with just one configuration option.
Typographical quotes with just 7-bit ASCII characters in the source.

However, there are algorithmic shortcomings for 2 reasons:

Dual use of the "ASCII-apostrophe" (') as single quote and apostrophe.
Languages that do not use whitespace around words.

So, please consider also Why You Might Not Want to Use "Smart" Quotes in Your Documents.

Escaping

The SmartQuotes transform does not modify characters in literal text such as source code, maths, or literal blocks.

If you need literal straight quotes (or plain hyphens and periods) in normal text, you can backslash escape the characters to preserve ASCII-punctuation.

Input	Output	Input	Output
\\	\	\...	...
\"	"	\--	--
\'	'	\`	`

This is useful, for example, when you want to use straight quotes as foot and inch marks:

6'2" tall; a 17" monitor.

Localization

Quotation marks have a variety of forms in different languages and media.

SmartQuotes inserts quotation marks depending on the language of the current block element and the value of the "smart_quotes" setting.[1] There is built-in support for the following languages:[2]

af:: "'Afrikaans' quotes"
af-x-altquot:: "'Afrikaans' alternative quotes"
ca:: "'Catalan' quotes"
ca-x-altquot:: "'Catalan' alternative quotes"
cs:: "'Czech' quotes"
cs-x-altquot:: "'Czech' alternative quotes"
da:: "'Danish' quotes"
da-x-altquot:: "'Danish' alternative quotes"
de:: "'German' quotes"
de-x-altquot:: "'German' alternative quotes"
de-ch:: "'Swiss-German' quotes"
el:: "'Greek' quotes"
en:: "'English' quotes"
en-uk-x-altquot:: "'British' alternative quotes" (swaps single and double quotes)
eo:: "'Esperanto' quotes"
es:: "'Spanish' quotes"
es-x-altquot:: "'Spanish' alternative quotes"
et:: "'Estonian' quotes" (no secondary quote listed in Wikipedia)
et-x-altquot:: "'Estonian' alternative quotes"
eu:: "'Basque' quotes"
fi:: "'Finnish' quotes"
fi-x-altquot:: "'Finnish' alternative quotes"
fr:: "'French' quotes"
fr-x-altquot:: "'French' alternative quotes"
fr-ch:: "'Swiss-French' quotes"
fr-ch-x-altquot:: "'Swiss-French' alternative quotes" (narrow no-break space, see http://typoguide.ch/)
gl:: "'Galician' quotes"
he:: "'Hebrew' quotes"
he-x-altquot:: "'Hebrew' alternative quotes"
hr:: "'Croatian' quotes"
hr-x-altquot:: "'Croatian' alternative quotes"
hsb:: "'Upper Sorbian' quotes"
hsb-x-altquot:: "'Upper Sorbian' alternative quotes"
hu:: "'Hungarian' quotes"
is:: "'Icelandic' quotes"
it:: "'Italian' quotes"
it-ch:: "'Swiss-Italian' quotes"
it-x-altquot:: "'Italian' alternative quotes"
ja:: "'Japanese' quotes"
lt:: "'Lithuanian' quotes"
lv:: "'Latvian' quotes"
nl:: "'Dutch' quotes"
nl-x-altquot:: "'Dutch' alternative quotes"
pl:: "'Polish' quotes"
pl-x-altquot:: "'Polish' alternative quotes"
pt:: "'Portuguese' quotes"
pt-br:: "'Portuguese (Brazil)' quotes"
ro:: "'Romanian' quotes"
ru:: "'Russian' quotes"
sh:: "'Serbo-Croatian' quotes"
sh-x-altquot:: "'Serbo-Croatian' alternative quotes"
sk:: "'Slovak' quotes"
sk-x-altquot:: "'Slovak' alternative quotes"
sl:: "'Slovenian' quotes"
sl-x-altquot:: "'Slovenian' alternative quotes"
sr:: "'Serbian' quotes"
sr-x-altquot:: "'Serbian' alternative quotes"
sv:: "'Swedish' quotes"
sv-x-altquot:: "'Swedish' alternative quotes"
tr:: "'Turkish' quotes"
tr-x-altquot:: "'Turkish' alternative quotes"

uk:: "'Ukrainian' quotes"
uk-x-altquot:: "'Ukrainian' alternative quotes"
zh-cn:: "'Chinese (China)' quotes"
zh-tw:: "'Chinese (Taiwan)' quotes"

Quotes in text blocks in a non-configured language are kept as plain quotes:

undefined:: "'Undefined' quotes"

[2]

The definitions for language-dependend typographical quotes can be extended or overwritten using the "smartquotes_locales" setting.

The following example ensures a correct leading apostrophe in 's Gravenhage (at the cost of incorrect leading single quotes) in Dutch and sets French quotes to double and single guillemets with inner spacing:

smartquote-locales: nl: „”’’
                    fr: « : »:‹ : ›

Caveats

Why You Might Not Want to Use "Smart" Quotes in Your Documents

For one thing, you might not care.

Most normal, mentally stable individuals do not take notice of proper typographic punctuation. Many design and typography nerds, however, break out in a nasty rash when they encounter, say, a restaurant sign that uses a straight apostrophe to spell "Joe's".

If you're the sort of person who just doesn't care, you might well want to continue not caring. Using straight quotes -- and sticking to the 7-bit ASCII character set in general -- is certainly a simpler way to live.

Even if you do care about accurate typography, you still might want to think twice before "auto-educating" the quote characters in your documents. As there is always a chance that the algorithm gets it wrong, you may instead prefer to use the compose key or some other means to insert the correct Unicode characters into the source.

Algorithmic Shortcomings

The ASCII character (u0027 APOSTROPHE) is used for apostrophe and single quotes. If used inside a word, it is converted into an apostrophe:

Il dit : "C'est 'super' !"

At the beginning or end of a word, it cannot be distinguished from a single quote by the algorithm.

The right single quotation mark character -- used to close a secondary (inner) quote in English -- is also "the preferred character to use for apostrophe" (Unicode). Therefore, "educating" works as expected for apostrophes at the end of a word, e.g.,

Mr. Hastings' pen; three days' leave; my two cents' worth.

However, when apostrophes are used at the start of leading contractions, "educating" will turn the apostrophe into an opening secondary quote. In English, this is not the apostrophe character, e.g., 'Twas brillig is "miseducated" to

'Twas brillig.

In other locales (French, Italian, German, ...), secondary closing quotes differ from the apostrophe. A text like:

.. class:: language-de-CH

"Er sagt: 'Ich fass' es nicht.'"

becomes

«Er sagt: ‹Ich fass› es nicht.›»

with a single closing guillemet in place of the apostrophe.

In such cases, it's best to use the recommended apostrophe character (’) in the source:

’Twas brillig, and the slithy toves

Did gyre and gimble in the wabe;

All mimsy were the borogoves,

And the mome raths outgrabe.

History

The smartquotes module is an adaption of "SmartyPants" to Docutils.

John Gruber did all of the hard work of writing this software in Perl for Movable Type and almost all of this useful documentation. Chad Miller ported it to Python to use with Pyblosxom.

Portions of the SmartyPants original work are based on Brad Choate's nifty MTRegex plug-in. Brad Choate also contributed a few bits of source code to this plug-in. Brad Choate is a fine hacker indeed. Jeremy Hedley and Charles Wiltgen deserve mention for exemplary beta testing of the original SmartyPants.

Internationalization and adaption to Docutils by Günter Milde.