Despite achieving the means to print in mass quantities, the publishing industry’s productivity was largely constrained by the need to set type by hand.
By 1880, the Linotype Printing machine was invented. The machine used a keyboard, which allowed an entire line of type to be laid at once. The requisite time and effort to lay out pages prior to publication were drastically reduced. As a result of this newfound efficiency, the number of communities that published their own newspapers nearly tripled.
A prime example of a well-preserved newspaper from the late 1800’s is this issue from The Cedar Rapids Gazette, dated January 10, 1883. This was the first issue of the Gazette, although other papers in Cedar Rapids preceded it. Even though it utilized the less-expensive wood pulp paper, it was well kept, and epitomizes a perfect specimen of a late 19th century newspaper using a “modern” rotary printing press for mass production.
When the paper was preserved to microfilm, some fading and effects of aging had begun; however, it was in far better shape than most papers of its era. It was slightly skewed at the time of filming, which can also pose a challenge to some OCR engines. However, in this example, the skewing is slight, and the overall quality is exceptional.
As you can see from the coverage, the majority of this page was “recognized” by the OCR software, and the “zones” were, for the most, part intact.
When the paper was preserved to microfilm, some fading and effects of aging had begun; however, it was in far better shape than most papers of its era. It was slightly skewed at the time of filming, which can also pose a challenge to some OCR engines. However, in this example, the skewing is slight, and the overall quality is exceptional.
As you can see from the coverage, the majority of this page was “recognized” by the OCR software, and the “zones” were, for the most, part intact.that» collision was inevitable. Tho purser of the City of Brussels states that at the last motnont they put on steam with tho view of moving to avoid the collision, but useless. As ihowiijg tho terrible nature of the collis- ion, tho purser says tlie main compiutiitout of the City of Brussels was cut right in two. After the collision the fog thickened, a.:id It was dangerous even for boats tu rnovn about They, however, approached the sinking vessel and found a boy named Hamilton, half drowu- e8, clinging to the mizzen cross trees. Tho two Italinna drowned seemed to have lo»t thoir wito from terror, and jumped overboard wilti- out waiting to s<« what could 1» il»ne for tlieir safot}'. The persons lost im-lude Connor, the quartermaster, and Cochraiie, iho steward noss in danger exemplified b j the captain, pur- b'u'liiiios, mid when thay begin to shrink somo ser and other officers of tbe ill-latod vessttl, ol1 them will l-ursttol, and notice of tlio number of debts which aneo, who was present at the first inauguration ou tho 'Ponobsuot, ta far back a» 1877. On Ilia at Allen Bros. "I'on lo.ik over this city, and you wm nnd .itloiiof her voyage." harillv any of Clio merchants or manufacturers Neither the passengers nor crew were ablo or «o'.nininiia:i men making anything. Go 0 BK.VO auvthius, and many pcrs'j.iS wero aronu:! tlnwo -
New York hotels, and inquire .is obliged to leave tho ship with but little ctotb- to ibei.- pui’coiituxes compared to two yours inz Thi-rnisnodoubtifthe”OKhadi-loareu ago. Thevwillalltollyoutb.-ittheyarenot 1 Stile so:.nci »1! would have boon saved. making mi) my us tlu-y wore, and tliucotiBn’.ner ThethuorvcoiK-ui-.iinxihe(In)i.niiig.’tihourn]v;.-itori.iVotr«in.-liiiig.Gointothestures -^ennioii is tha: whim tin- vejsel mr,-netl they just iii’w, al’tor tlirt holiilnys, and see how were struck by the rigging ami rendered in- many incivha.its ti-11 you they have had a sensible. Ou» man was found dead in me ni-MH|K-rous Christina*. Tho mercantile estftb- Iisbintfiii-f hi thi* city and throughout tho ‘ Many of the Brussels passengers aro suffer- i-oiinli-v are i-rimniHil with goods. Tho ing greatly fron? thu shock. liixiil:-,-; the faclnnlis rairioi ili^-piwi) of what they innice, ;n)r the- tnurcluints of what tlmy huld.’ f io into Wall ntri-ct among the eleven liiiiiiln-.l br.ik.si-s tliure, and ask how many of r’u-ni aro tnaki :g unv money. Thorn is no public in tin- rtlruu T’he ntodis aro kept up tn tbi’ir visionary valuer by manipulation.
Latuer details of the disaster say C»pt. Land was one of the first picked up by tlm boats, al- Uiongh bo, with tbo’obief euL-incor, had Iwen tha last to leave ‘ie rigging. Wlion rc/«uiiecl lie was in the act of passing life buoy to realized ‘.’or a few years past Wo have every- drowning comndo.
As evidenced by the blurred text, paper flaws and skewing of the page, there is still a considerable number of OCR “mistakes” in this excerpt. However much of the important information remains intact. If the “non-words” were stripped from the OCR text output, there would still be left with enough “real words” to piece together the story, as seen below:
that collision was inevitable. purser of the City of Brussels states that at the last they put on steam with view of moving to avoid the collision, but useless. As terrible nature of the collision, purser says main of the City of Brussels was cut right in two. After the collision the fog thickened It was dangerous even for boats about They, however, approached the sinking vessel and found a boy named Hamilton, half clinging to the cross trees. two drowned seemed to have from terror, and jumped overboard out waiting to what could The persons lost Connor, the quartermaster, and steward in danger exemplified the captain, mid when begin to shrink and other officers of them will, and notice of number of debts which , who was present at the first inauguration, far back 1877. On at Allen Bros. over this city, and you her voyage.” any of Clio merchants or manufacturers Neither the passengers nor crew were amen making anything. and many -New York hotels, and inquire .is obliged to leave ship with but little to compared to two yours u ago. Stile so! would have boon saved. making my us wore, and is whim they just, and see how were struck by the rigging rendered in- many you they have had a sensible. man was found dead in me Christina mercantile city and throughout Many of the Brussels passengers with goods. greatly shock. the of what they the- of what into Wall among the eleven and ask how many of money. Thorn is no public in kept up visionary by manipulation. details of the disaster say. Land was one of the first picked up by boats, with, had last to leave rigging. was in the act of passing life buoy to realized ‘.’or a few years past have every drowning
With the exception of some text that is clearly out of place given the context of the article- specifically,the words related to the Allen Bros debts back in 1877-, the grouping of key words and phrases render this article much more search-able than other examples.
The paper stock used for this page, in 1883, predates microfilming practices for the preservation of newspapers by 50 years. Although this example seemingly represents a very well-preserved paper, the source material and time of filming posed the same issues that burdened newspapers filmed 100 or more years prior. Unfortunately, many of the newspaper pages that were preserved in the 1830’s, 1840’s and 1850’s did not make it to the preservation cameras in nearly as good condition.
The Saturday Evening Journal from Muscatine Iowa, published in 1888, shows a significant improvement in quality.
This newspaper is also in notably good shape, and the quality of the image is outstanding. This is another example of a “best case scenario” for such an old document, especially for a newspaper of this era.
The top quarter of the Saturday Evening Journal alone illustrates the challenges that linger with historical newspapers from the late 1800s, even when they are preserved so meticulously as in this example, which is a rare occurrence.
Given the high quality of both the source material and the digital image, the OCR software was able to “read” nearly 930 words in the top quarter of the page. The count alone leads one to believe the OCR is quite good, but the issues inherent to OCR-ing old newspapers persist. If the words on the page were manually counted, one would discover that nearly 300 words were not captured at all. This may be due to faded text, smudging on the page or the proximity of the words and layout of the columns.
Further investigation of the OCR leads to additional discoveries regarding the OCR quality. Much like the text on the page, the OCR output is dense. The text output for the just a few lines of the image above, starting from the top, is as follows:
DAILY JOURNAL. buque Herald a commendation of Com
ruff, a former member of the board, J. Lundy, of Detroit, read a paper on within whose borders Salt river is to Examples of ideas which seem to be So Serious Resistance Off red, Ie* The New York Witness pertinently “Hygiene in Relation to the Eye,’ be found. The river or creek is rapid, indigenous to all countries occur to the Reason to Expect It.
MUSCATINE, IOWA, SATURDAY, NOVEMBER 24.1888. The Care of the Eyes. Exchange: The origin of Salt river SIMILARITY PROVERBS.
PRICE FIVE CENT PROGRESS OF THE IOWA EYiC- TIONS* At the sanitary coBvention held at or the imaginary hard road to travel Idea# Which Seem to Be Indigenous to missioner Dey from Marcus A. Wood Ann Arbor, Mich., not long ago, Dr. (J. for office-seekers is traced to Kentucky All Countries—Examples.
engaged with railroads for the past ten eye troubles is shown to be the gation. It was common in early days man and Russian. In the sunny south States Marshal Forsey and his falce erf present duty of prohibitionists is clear. excessive strain upon the muscles in Kentucky to consider a “row up Salt it takes the form, “One flower does not deputies evicted four Dos Moines rivw It is to revise their plan of campaign years, being now prominently connect and nerves of the eyes due to faulty river” as the synonym of hard work. make a garland.” In Italy we find land settlers at Homer, Hamilton coun
The first thing you will notice is that the order of the words is inconsistent with the image. The headline is omitted, and the mast line is inserted after text located further down in the page. Each line of text fails to provide context for the next.
This is due to the software’s failure to properly detect the columns and “read” the page like a book. The software scrutinizes the top left corner of the page, starting with what it detects as a word, and then reads across. If it comes across a “zone” in the process, it will insert that section where it was detected.
Although the Linotype newspaper print a reads clearly to the human eye, it is challenging for a computer to organize the data without context.
Additionally if you were to “clean” the OCR text output to contain only real words, then the 930 “words” that were initially captured are reduced to 852 words. That is still over 70 percent “real words” and a far better result than earlier papers. The search-ability for “keywords” increases dramatically, but the search-ability of “key phrases” is still severely limited.
Using the actual image as a guide, and parsing out the OCR into logical groupings, the chaotic arrangement is apparent from just the first few hundred words:
DAILY JOURNAL. Herald a commendation of a former member of the board, J. Lundy, of Detroit, read a paper on within whose borders Salt river is to Examples of ideas which seem to be So Serious Resistance Off red, The New York Witness pertinently “Hygiene in Relation to the Eye,’ be found. The river or creek is rapid, indigenous to all countries occur to the Reason to Expect It. MUSCATINE, IOWA, SATURDAY, NOVEMBER 24.1888. The Care of the Eyes. Exchange: The origin of Salt river SIMILARITY PROVERBS. PRICE FIVE
CENT PROGRESS OF THE IOWA
Clearly the “block of text” captured contained fragments from 10 or more “zones” on the page; it combined most, and omitted others. The headline is not interpreted as text at all. When a row of text does not line up to the next line of text in an adjacent column, the software attempts to determine which word is “next”. If the software does not detect the column, it will move on to the next word in the closest proximity. If the columns are out of alignment, either a line of text above or a line of text below may be omitted.
To help put this in perspective, the fragments can be compared to the image and manually grouped together to create this:
MUSCATINE, IOWA, SATURDAY, NOVEMBER 24 1888 PRICE FIVE CENT
DAILY JOURNAL The New York Witness pertinently ,
The Care of the Eyes “Hygiene in Relation to the Eye,’
Herald a commendation of a former member of the board, J. Lundy, of Detroit
SIMILARITY PROVERBS.
Examples of ideas which seem to be be found.
So Serious Resistance Off red
the Exchange The origin of Salt riverThe river or creek is rapid, indigenous to all countries occur to: , Salt river is to
read a paper on within whose borders
Reason to Expect It
PROGRESS OF THE IOWA
If this process were to be repeated nine more times, you would have the 850 words organized into 11 or more groupings. These structured groupings would allow for much more refined searching and indexing of the content. Since the data is unstructured to such a high degree, the software cannot be trained to “learn” these patterns and there is no contextual basis for the computer to know how to combine the words and letters into something comprehensible.
If the “zoning” algorithms were refined and the rules regarding detection parameters were applied for stricter definitions of the beginning or end of a column, row, or sentence, it would create more chaotic OCR output. Breaks and zones could be applied by spacing inconsistencies or unintentional detection of a column by an impurity in the paper, a printing defect, or a scratch on the microfilm from which the image was captured. This has lead to additional utility issues as the software applies a “zone” anywhere it detects a defined structured data set, which causes the text output from OCR to look like this:
DAILY JOURNAL. The origin SIMILARITY PROVERBS Herald a commendation of Salt river of a former member PROGRESS OF THE IOWA of the board, IOWA be So Serious SATURDAY, read a paper on J. Lundy, of Detroit, seem to Resistance to the Eye,within whose borders
The New York Exchange be found. MUSCATINE, The river or creek is rapid, Witness pertinently The Care of the Eyes. Salt river is to Examples of ideas which Off red, NOVEMBER occur to the “Hygiene in Relation indigenous to all countries
Reason to Expect It. 24.1888 .PRICE FIVE CENT
Due to the lack of structure, the wild inconsistencies in alignments, and the poor quality of the source material or preservation mediums, the amount of “intelligent” machine-driven OCR is limited. Defining zones manually would be necessary, but cost prohibitive.
This is an abrupt transition back to the historical timeline, and the procedural details are starting to get hefty and repetitive. May be good to take a break here and save the rest for the next article.
In 1889, the Monotype machine was introduced. The machine functioned much like the Linotype machine, as monotype operator similarly typed out a text. One advantage of the Monotype was that it was easier to correct a misprint; a single letter of type could be removed, thus eliminating the need to recast a whole row of type. Monotype also produced a finer quality type. As a result it was frequently used in the book trade, while linotype was often used at newspaper presses due to its speed and economical benefits.
To counter the rapid deterioration of the newsprint original and the numerous difficulties in the storage and use of newspapers, the New York Times began to archive their papers on microfilm in 1935.
