Line numbering: an unexpected test

A most unexpected instance of this situation arose immediately after the initial transcription of Ht, at the stage in which line numbers are typically assigned to manuscript transcriptions in the PPEA. In the PPEA, two sets of line numbers are assigned to each manuscript line: one, encoded on the id attribute, indicates the absolute position of the line in the relevant manuscript; and a second, usually assigned automatically by a PERL script to the n attribute, identifies the equivalent line in the relevant Athlone (print) edition. Thus for example, line 50 of the Ht Prologue is line 50 of the manuscript poem and can also be affiliated confidently with line 50 of the Athlone edition of the B version of the poem (indicated here by the sigil KDP):

l id="HtP.50" n="KDP.50" I sawe somme þat seid þei had soght seyntes/l

In most PPEA editions, assigning this canonical line number has proven to be a relatively straightforward task. In a typical manuscript, the scribe copies his text from an exemplar or exemplars representing a single state of the poem—that is, an exemplar clearly in the A, the B or the C tradition—and the affiliation of individual lines is quite predictable. The main exceptions are the so-called splices, in which a manuscript combines large sections drawn from the A text with another large B- or C-text section.

Ht, however, is different from these other witnesses in that its mixing of text from the A, B, and C traditions is thorough, unpredictable, and occurs at the level of long passages, single lines, half-lines, phrases, and even single words. As a result, any given line, word, or phrase in its 8620 lines might come from any of the three versions of the text, arbitrarily.

For the purpose of the present discussion, the additional possibility of a line or smaller reading being related to a genetic group or whole tradition no longer extant will be left aside.

Determining which tradition or traditions a given passage is most closely affiliated with can be accomplished only on the basis of a line-by-line comparison with the extant texts, largely in the hope of finding readings unique to the A, B, or C tradition that also appear in Ht. Localization of these variants—referred to hereafter as discriminant variants—then makes possible the discovery of the filiation of at least some of the Ht scribe's exemplars. Since many lines in the poem are found in substantively identical or near identical forms in more than one version, the affiliation of many passages can be made only tentatively on the basis of such discriminant variants in adjacent lines.

This process obviously requires a more thorough collation than the preliminary scan that has thus far proved sufficient for all other manuscripts in the PPEA. As a result, at the earliest stage of the markup, the edition of Ht was able to identify passages only in terms of their absolute position—supplying a value for the id attribute, but leaving the intra-textual reference n empty:

l id="HtP.1" In a somer sesoun whan softe was þe sounne/l

Collation by approximation

The question then remained: how best to discover, if possible, the filiations of the exemplars of Ht? Since too few of the electronic transcriptions of the surviving manuscripts had been fully proofed, each and every reading of Ht—amounting to about 88,000 words, together with an incalculable number of word-order and omission variants—would somehow have to be collated against all three Athlone texts (representing something over 200,000 words of comparative readings). Thereafter, Ht would also have to be collated against the not inconsiderable number of variants recorded only in the three Athlone apparatus—apparatus in which, for the reasons stated earlier, Ht itself does not appear. Doing either or both of these collations in a straightforward way, word by word and phrase by phrase, would have been prohibitively time-consuming and most likely would have abounded in error beyond any acceptable level.

Were this to have been the only available method, it might have resulted in either scrapping the project, or giving up on using n to provide canonical references to the Athlone editions. Since such a measure would almost completely vitiate the goal of discerning the scribe's likely exemplars—one of the project's principal warrants—a simple method of discovery in easy stages was undertaken. Since completed and proofed transcriptions of all the surviving witnesses to A, B, and C were not available for machine collation, the method had to be based on the Athlone print editions while minimizing wherever possible both work time and the multiplication of error.

Identifying parallel lines

The first problem was to discover simply whether a given line appeared in a single tradition, two traditions, or all three. The first line of the Prologue, cited above, appears in all three traditions, making it a line that might draw its variants from a manuscript related to any one of the other surviving manuscripts and fragments of the poem. Other lines (the first at HtP.7) appear in only two states of the text, greatly limiting the number of manuscripts to which Ht can be related at this stage in its text. Finally, and most usefully of all, there are those cases in which a whole line itself will be a discriminant variant, appearing in only one tradition, thereby narrowing the field of possible affiliation with surviving manuscripts significantly. In these cases alone (the first such in Ht at HtP.50) can a value be assigned with confidence to the n attribute as used by the PPEA without detailed collation:

l id="HtP.50" n="KDP.50" I sawe somme þat seid þei had soght seyntes/l

Certainly, marking up such lines, dubbed unaries, according to TEI and PPEA recommendations would prove useful for later analysis of the filiation of Ht, since any strong patterns of affiliation could be better relied upon not to have been the result of contamination from the other traditions. Nevertheless, if the number of such lines proved very small—as in the case of the A-text unaries which number only 143—one might wish to supplement this data set with those binaries and ternaries that proved to have discriminant word- or phrase-level variants that would identify them as affiliated with only one tradition.

Encoding the collations

How might one record these sets and subsets in order to make them machine-readable without straying unduly from TEI and PPEA markup standards? Each of the three methods for constructing a reference system described in the TEI recommendation posed problems in the case of Ht. Simply declaring a reference system by the prose method (P4 5.3.5.1 and P4 6.9; P5 5.26.80 and P5 6.40), and then scattering the needed references to all three Athlone editions into various newly declared attributes in each l element would have required changing the SEENET DTD and would also have required keeping track of the self-defined data structure on the fly. The milestone method (P4 5.3.5.3 and P4 6.9.3; P5 5.26.80 and P5 6.40.105) presented the further drawbacks of clashing with the current use of milestone by the PPEA to encode folio breaks.

The TEI ref element seemed a possibly good choice, since it defines a reference to another location in the current document, in terms of one or more identifiable elements, possibly modified by additional text or comment (P4 35 ref; P5 35 ref) and its model could be adapted to the data needed for study of Ht. Its use, however, would not be without some complications. Because this element, like the other two TEI-recommended methods of constructing a reference system, was designed as a means of encoding references to clearly known locations, it naturally does not provide for the use of a cert attribute. At this initial stage, however, most identifications could be only hypothetical, pending examination of the word- and phrase-level variants. Even when this examination would be finished, moreover, many lines—perhaps the majority—would remain forever ambiguous in that they would be found to contain no variants present in only one tradition of .

Such a reference system is really more a matrix of hypotheses at first, and, if one is lucky, a matrix of hypotheses together with somewhat more and somewhat less clear-cut determinations in the end. Since the TEI recommendation does not and was never intended to take into account such high levels of ambiguity, none of the suggested reference systems would accurately and completely encode what would be known (or more accurately, would not be known) about the filiation of binary or ternary lines at the point at which they had been indeed identified as binary or ternary, yet before their internal variants had been thoroughly examined to find the discriminants that, if present, could identify them with a single tradition or with an even smaller group of manuscripts.

Adding a cert attribute to the recommended reference systems or to the ref element under P4, using the new certainty element of P5 (), or conducting a final collation of all variants would not resolve the issue. Even at the end of all the identification of variants it remains unclear what would be certain, and the degree to which it would be certain. What percentage of certainty could be assigned—that is to say with any rigor—to a ternary line with two B-text discriminant variants agreeing with Ht and another different variant from the C-text, but with none from A? Although a real statistical analysis of all the variants witnessed by surviving manuscripts set against Ht would be possible in theory, such an analysis would require that the whole work of identification of variants be completed and encoded for machine analysis first, exceeding by far the practical time and financial constraints on the project.

As the recommendation for certainty in P5 states (), the assignment of a level of probability using the new element, as with the cert attribute of P4, typically remains subjective. This is no problem at all in a project in which some dozens or even a few hundred instances would be marked up this way. The scale both in number of instances and in the time needed to complete each stage of the project presented by Ht, however, required a more categorical method, one that would be transparent at later stages of the work and also easily grasped by new minds—semi-skilled proof-readers, for example—grappling with decisions made in the course of collating thousands of lines across three versions of the poem perhaps a year or more after initial encoding. The more unambiguous such decisions could be made, the better.

A practical method for indicating whether a line had zero, one, two, or three apparent parallels in the A-, B-, and C- text traditions was therefore developed for use on the fly during the line-level identification stage. This would serve as a guide at the later stage during which variants would be examined individually, based on the preliminary determination of their range of possible sources in the A-, B-, or C-texts. Such an approach would not guarantee that the correct determination would be made in every case, but it would make it much easier to discover whether or not it had.

The n attribute of TEI l was already being used by the PPEA to record an absolutely certain reference to line numbers in the Athlone editions and thus was unavailable for a different use in the Ht. For legacy reasons, the project also hoped to avoid modifying the PPEA DTD in order to prevent the breaking of browsers and other software and to avoid as well the need to supply a special DTD—and a non-conformant one at that—for a single text within a larger collaborative project.

A further, entirely practical, consideration at this stage was equally pressing. Some way of recording this data that would be very compact and easily referenced by eye as work progressed through the manuscript's 8620 lines, across three print editions and several other references, had to be developed in order to minimize as much as possible the likelihood of human error. The human eye may easily skip while focusing on just one text, as witnessed in the work of all the descendants of Adam Scriveyn to this very day, not excluding electronic editors. Normally a text is marked up with the eye trained directly at one screen or at most moving from the electronic transcription to a digital image or to a single standard print edition. The chances of eye skip producing an error while an editor scans seven or more reference sources, all in eccentric spelling, are clearly multiplied. For this reason, it seemed preferable to develop a simple symbolic code, suitable for use as an attribute value, that could be used to reflect basic collation decisions.

To avoid changing the DTD, the global attribute rend—as yet unused by the PPEA—was pressed into service at this early stage. This was a deliberate act of attribute abuse, since rend is intended to encode how the element in question was rendered or presented in the source text (P4 33 Element Classes: global ; P5 Element Classes) and the Ht project was at best using it to describe how elements might be distinguished in the output. The alternatives, however, seemed likely to cause much more damage to the project within its wider PPEA context, and this use of rend could itself be carefully documented elsewhere so as to conform to TEI recommendations at least at the lowest level of conformance.

The line-level identification itself was done in three passes—one against the A-text, one against the B-text, and one against the C-text—minimizing the need to shift between printed reference sources and recording only the absence or presence of a line in each version of . A system of symbols was devised to account for the unary, binary, or ternary state of a line as that state gradually became apparent. Additional notation was devised to represent whether or not a given line could be upgraded to a unary status by later identification of discriminant variants, and also to represent clearly which of the three traditions the line could possibly represent—in distinction to that tradition which it would be finally determined to represent. This value was then recorded in the n attribute of l. Thus, for line 62 of the Ht Prologue, the rend attribute after the initial approximate collations were completed contained the following symbols, where % represents the presence of a line in one tradition and [] contains a value of A, B, and/or C, indicating the tradition or traditions in which a parallel line initially was discovered:

l id="HtP.62" rend="%%%[ABC]" Meny of þes maistre freres may cloþe hem at lykynge/l

After this line-level identification, the potential parallels could be examined in the hope that discriminants would emerge—as indeed is the case in line 62, where Ht shares the reading maistre freres only with B-text manuscripts W Hm Cr1 CR2 Cr3 Y O C2 C L M H. As a result of this subsequent examination, the line also receives a +—a symbol used to identify the line for subsequent processing and addition to the pool of unary lines. To such an upgraded binary or ternary line and to all unary lines, an n attribute value could then be supplied with a high level of confidence; this value represents, in the case of unaries, the unambiguous assignment of the line to the A-, B- or C-text tradition:

l id="HtP.62" n="KDP.62" rend="%%%+[ABC]" Meny of þes maistre freres may cloþe hem at lykynge/l

All other lines, whether binary or ternary, would always have to be examined against the two or three traditions to which they might be related, and any determination of their line-level affiliations would have to be made based on the affiliation of the surrounding unary and upgraded lines. It would be only at this late stage that any sort of rigor could be applied to the assignment of certainty values.

Clearly, in the course of this inductive process, when the value of n could finally be assigned with some confidence, and still more when the full reference system had been constructed, the compact encoding in rend became redundant. On the way toward this goal, however, it proved invaluable for its compactness, for its usefulness as a guide to the status of lines in mid-process (when only the A- and B- texts had been examined), and as a way to delay committing to a more elaborate reference system (and perhaps additionally having to break existing display tools) until it was certain which system might be the most appropriate.

Occupying not more than the space of nine characters in their encoding, these values for rend could be kept precise and represent known levels of certainty. They are easily processable and renderable in order to enable unskilled or semi-skilled assistance in the later proofing of the matrix of parallel line number references. By using this categorical approach, the project side-stepped problems with the ad hoc determination of certainty by the encoder. Each step in the process represents a defined degree of certainty and project staff working with the text in the future have an easily understood and very clear representation of the collation process used and the reliability of its current state.

In this case, abuse of rend seemed to represent both the least damaging and the most transparent way to encode crucial information about the relation of Ht to the A, B, and C traditions of as that information built up gradually. It can of course be easily re-encoded as an entirely new and different attribute later. The particular situation involving heavy contamination from many manuscript traditions may be comparatively rare, at least among manuscripts of , but it is not by any means unique to Ht, nor, surely, is the situation rare in which something ambiguous in a document needs to be transcribed and marked up as a means of eventually discovering its nature through subsequent processing—which may in turn result in a subsequent, more appropriate markup.

Experimental markup as part of a recommendation

TEI and PPEA recommendations tend for very laudable reasons to address the most common instances in which the significance of a text string is already known or at least easily discernible before markup commences. Indeed, one of the chief aims of the TEI is to make available a standard of interchange for those aspects of texts which are of recognized importance to the widest possible community.

The technologies applied by and applicable to TEI-encoded texts, however, show great promise for the gradual examination and elucidation of as yet unknown or imperfectly known phenomena in a manuscript. A globally available attribute for the encoding of speculative or experimental data of the kind described above would foster rigorous experimentation within a controlled context without deliberate tag and attribute abuse. An x, exp, or even temp attribute would perhaps seem to some to allow for undisciplined markup habits; in actual practice, however, such attributes would foreground the experimental or provisional nature of their values, thus in fact lending both rigor and transparency to a process that under certain circumstances is crucial to making a more determinate decision later—one that might be much more suitable for encoding under current recommendations.

Provision of avowedly experimental attributes for existing elements would also support the production of editions of manuscripts containing non-standard features under the real conditions of large electronic projects in which there is usually a significant lag time between the discovery of an anomaly that needs to be marked up in a way not provided for in existing recommendations, the final decision by an editorial board as to how to handle the phenomenon in question, and the retrofitting or new development of ancillary software to support the change. Having available experimental place holders that are already provided for in the standard DTD would allow controlled experiments to be carried out without breaking all or most existing support mechanisms at a stage during which it is uncertain that major changes should be made to accommodate the experiment on a permanent basis, as a received standard of proven merit.

Inverse recension using TEI app

Although it was hoped that the Ht project could conform to all TEI P4 and PPEA recommendations from this point forth, it became abundantly clear at the very next stage—that of the search for discriminant variants—that a second extension would have to be undertaken, that of adapting TEI app to the encoding of witnesses to variants in a documentary edition in which the forms of the variants themselves would be less important than their simple agreement or disagreement with the readings of Ht.

When each line in Ht had been shown either to be Ht-unique, or parallel to one, two, or three of the traditions of , examination of the variants could commence. At first a rapid reading against the three Athlone editions was undertaken, chiefly as a means of disambiguating whether a binary or ternary line could be seen to have come from only one of the three traditions by discovery of discriminant variants such as maistre freres, a reading unique to the B-text, as in the earlier example from Ht Prologue, line 62.

After this final preliminary stage, serious collation of all the variants and recording of the sigils of all witnesses bearing them could finally begin in order to discover if any relationship existed between Ht and the known genetic groups of surviving manuscripts.

The process, however, was vexed from the start. The encoding of the lemmas of Ht set against the readings of surviving witnesses was not possible using a straightforward application of TEI and PPEA recommendations. In this case, the PPEA did not at the time use TEI app at all, since editions in the initial PPEA series are documentary, building together into a database of all the readings of all the manuscripts, to be sure, but reserving the majority of such inter-textual analyses for the later stage in which archetypes would be reconstructed.

Analysis of Ht's genetic relationships, leading to the discovery of the filiation of at least some of its exemplars, however, could not be at all efficiently done without the use of computing. Typically in the PPEA, a manuscript is read against the recension from which it derives, with special attention given to its agreements with only one or two other manuscripts, or to somewhat larger, known genetic groups. These relationships, explicitly documented in the Athlone apparatus, are then recorded in notes for later collation and analysis in a narrative introduction.

Experimental use of TEI app is underway in the forthcoming edition of manuscript Hm (Huntington Library Ms Hm 128, ed. Hoyt N. Duggan, Michael Calabrese, and Thorlac Turville-Petre), though in this case it is used to examine only the Hm unique readings and those genetic groups already known to be of interest. Nevertheless, using this method, the Hm lemmas and the sigils of the manuscripts to which they are related need be encoded only once for use in multiple places and for multiple purposes throughout the Hm edition, something that was not possible with the use of notes inside note elements.

Since the relationship of Ht to the other manuscripts is so much more complex, and since the readings of Ht were not explicitly recorded in the Athlone editions because of its eccentricity (Kane and Donaldson 1988, 15), a means of markup had to be devised in the hope that some means of analytical display could be used to aid in the search for patterns of agreement in as inductive and machine-processed a way as possible—something that could not be done using the discursive textual notes of other PPEA editions, since these would have to be examined later by eye, one by one, in order to discover emerging patterns intuitively, or by constructing one chart or graph after another by hand.

The TEI app element was thus pressed into service for a sort of inverse recension. That is, unlike traditional recension in which a manuscript is chosen as a copytext and others are collated against it to weigh the merits of their various readings and discover as much as possible the readings of a hypothetical earlier state of the text, the goal in this case would be to see how other genetic groups fed into the hypothetical text that had already been constructed by the Ht scribe—the Ht manuscript itself. The Ht scribe's construction of a gesamt stands before us in the artifact itself. What lies hidden are the traces of the exemplars from which he constructed that text.

Although the TEI app element was originally intended to be used as a means of representing the information contained in the standard apparatus criticus of a full scholarly edition collating and weighing the readings of many manuscripts, this element was admirably suited as a data model to use in this inverse manner, even though the rdg element in this case would never have content meant to be interpolated into Ht as a better reading than that already witnessed by Ht. The rdg element would always record only those manuscripts and their readings that disagreed with Ht, while lem in turn would contain in its wit attribute only the reading of Ht, together with the sigils of any manuscripts that agreed with it. The content of rdg certainly would form a most useful database for later characterization of the nature of the variants in Ht set against other, less eccentric manuscripts. Indeed, a contrastive analysis of his choices might shed light on the scribe himself, his likely profession, class, level of education, and so forth. At this second stage of the work, aimed chiefly at the determination of the scribe's exemplars, however, the content of rdg was relatively unimportant, as long as the wit values were always accurately supplied.

Unique readings or even unique lines could in any case be foregrounded for a machine-generated database of readings that are arguably more likely to represent the scribe's own language because they would have a lem with a wit value of Ht only, and thus could easily be referenced by a stylesheet. Time constraints set against the very time-intensive nature of TEI app tagging and the very irregular nature of medieval spelling made it advisable to supply only a set of variables in place of the actual readings of disagreeing witnesses, as in the following example from Ht5.163, where there is very great divergence in readings among the surviving witnesses:

app type="variables" loc="Ht5.163" lemtresoexpanur/expan &~ tresouexpann/expan be not/lem  rdg wit="Cr1 Cr2 Cr3 Y O C2 C Bm Bo Cot L M H" type="s"{gamma}/rdg  rdg wit="F" type="s"{delta}/rdg  rdg wit="R" type="s"{epsilon}/rdg   rdg wit="Hm" type="s"{zeta}/rdg  rdg wit="W" type="s"{eta}/rdg  rdg wit="{sigma}" type="s"{theta}/rdg /app

Here, with the decision having been made to delay supplying the full set of readings for each disagreeing witness, each distinct reading is represented instead by the name of a Greek letter inside curly braces within its own rdg element, following each time the order in which they appear in the Athlone apparatus, in the hope that perhaps one day an electronic version of the Athlone editions could be used to supply the very eccentrically spelled readings in their place.

The final rdg in this usage always represents the reading of all manuscripts whose sigils are not already recorded in the current app, a necessary measure to take into account the fact that some sigils and their associated readings are, unfortunately, implied rather than specified in the Athlone apparatus. When app tagging is complete, an application developed by Shayne Brandon at the University of Virginia Institute for Advanced Technology in the Humanities examines all sigils present in the current app and replaces the value {sigma}—a variable customarily used by PPEA editors to represent all other manuscripts—with any sigils that may remain unaccounted for. Sometimes, as in the example above, the reading represented by the final rdg value (here {theta}) happens to be Kane and Donaldson's conjectural emendation, since all other B-text sigils are already in fact accounted for. In this case, then, the value {sigma} would not be replaced with any sigils, and this final rdg could be either expunged or used for a later examination of Kane, Donaldson, and Russell's emendations, independent of the Ht project.

As at all junctures in this project, a willing suspension of curiosity about the readings of Ht set against the other surviving manuscripts had to be maintained in order to keep the project within any sort of bounds of time and labor. The app-tagged unary lines, even in the absence of the variant readings, have made possible the development of XSLT stylesheets that extract and display the unique word- and phrase-level readings of Ht in relation to the sigils of agreeing and disagreeing witnesses, as well as the whole lines that are unique to Ht (including two very intelligently made unique Latin interpolations). This use of TEI app has also enabled the development by Doug Chestnut of the University of Virginia's Alderman Library of JavaScript, CSS, and SVG-based tools for examining patterns of filiation—tools which, though still in an early stage of testing have strongly pointed to the presence in certain passages of sub-linear contamination from each of two genetic groups of the B-text tradition, FH (by contamination) and OC2, suggesting the presence of two B-text exemplars at some points in the construction of the Ht text:

The Watersheds Diagnostic script applies background color to readings in Ht according to the user's selection of conditions fulfilled by the values of the wit attributes of lem and rdg. The script can be used to examine levels of contamination from one or more different witnesses as well as across the A, B, and C traditions.

Non-textual features in Ht

The need to establish, wherever possible, corroborative evidence for tracking the Ht scribe's shifts from one exemplar to another, and the need to demonstrate if possible that Ht is itself the original manuscript supplying its conflated text, motivated another type of markup that appears in neither the TEI or PPEA recommendations: the markup of phenomena that fall into a category of scribal paratext or scribal apparatus, such as corrector's crosses (whether erased or not, and whether or not they seem to have invoked an actual correction), other crosses used to indicate such activities as planned but delayed rubrication, marginal ticks, carets, guide words or guide letters, and marginal or inline scoring other than the plummet used for the ruling. Ideally all of these, in addition to all the ink stints and certain codicological features such as watermarks would be encoded so as to be available for display and analysis in relation to one another, and in relation to both the textual and codicological divisions and features of the manuscript in varying diagnostic views. Such views could, for example, reveal the typical length of the scribe's correction stints and, if set against the erased letter forms and the forms that replaced them, could also reveal their association with one or more exemplars used as proof text, or the association of a particular paper stock with such a shift in exemplars.

Many of these features are quite naturally beyond the scope of the TEI, which is a Text Encoding Initiative, rather than a Codex Encoding Initiative or a Scribal Paratext Encoding Initiative. Nevertheless, evidence of scribal planning and method for correction, ordinatio, and mise-en-page, whether planned only or both planned and executed, relates directly to the scribe's use of his exemplars. As a result, the ambition to discover the number and nature of these exemplars required at the time some sort of true extension to TEI P4, or at least that tag abuse would again have to occur.

Lessons learned