Semiotica 146 (2003), 69-79.
In this study, we describe the semiotic grammar involved in the reading of radiographic images. As in the case of natural language, this grammar has a morphology and a syntax (cf. Ducrot and Todorov 1972: 51). Semiotic morphology deals with the formal conditions satisfied by individual Roentgen signs (cf. Cantor 2000) and is based on the categorical typology of signs created by Charles S. Peirce. We will demonstrate the relation between Peirce's general typology of signs and the pragmatic typology of signs used in Roentgen diagnosis (cf. Cantor 2002). Semiotic syntax deals with the rules by which Roentgen signs are combined in the process of diagnosis. These rules or semiotic operators will be classified and examples will be given of their use in the reading of radiographs.
In this section, we briefly review the general typology of triadic signs created by Charles S. Peirce. This typology is generated by the three fundamental categories of thought as conceived by Peirce and termed by him Firstness, Secondness and Thirdness. In this application, the Peircean categories will be thought of as attribution, opposition and mediation, respectively.
A Peircean sign is a triadic relation between correlates termed representamen, object and interpretant. The representamen is an attribute of the object, the representamen is in opposition to the object and the interpretant mediates between the representamen and the object. For each correlate of a triadic sign there are three categorical correlates that determine three sign types (cf. Liszka 1996: 20, 21, 24). In what follows, Peirce's intuitive terminology for the categorical sign types has been adapted for use in Roentgen semiotics. A diagram of the Peircean typology of oppositions between representamen and object is given in Figure 1.
Representamen types
Three categorical types of representamen determine three types of Peircean signs termed qualisign, sinsign and legisign. The representamen of a qualisign is an attribute derived by prescissive abstraction (cf. Peirce 1905: 352) from the representamen of a sinsign. The representamen of a sinsign is an instantiation. The representamen of a legisign is a generalization obtained by hypostatic abstraction (cf. Peirce 1905: 352) from the representamina of sinsigns.
Opposition types
Three categorical types of opposition between representamen and object determine three types of Peircean signs termed icon, index and symbol. For an icon, the opposition has the attribute of commonality, meaning 'being the same in some respect' (likeness). For an index, the opposition is the relation of accompaniment, meaning 'occurring together in fact' (contiguity). For a symbol, the opposition is mediated by a convention, meaning 'an agreement between interpreters.'
Iconic types. Three categorical types of icon are termed image, diagram and metaphor. An image is an icon with commonality of appearance. A diagram is an icon with commonality of structure. A metaphor is an icon with commonality of meaning.
Indexical types. Three categorical types of index are described as referential, existential and nominal. A referential index involves the awareness of presence or absence, meaning 'accompaniment by attention.' An existential index involves entailment or 'accompaniment by necessity.' A nominal index involves naming or 'accompaniment by a word.'
Symbolic types. Three categorical types of symbol are described as abstract, singular and general. An abstract symbol involves a convention with an attribute as object. A singular symbol involves a convention with an existent as object. A general symbol involves a convention with a generalization as object.
Interpretant types
We assume that interpretants present in a natural language. Three categorical types of interpretant determine three types of Peircean sign termed rheme, dicisign and argument. The interpretant of a rheme is a phrase that expresses an attribute of its object. The interpretant of a dicisign is a statement that expresses a fact about its object. The interpretant of an argument is a pattern of inference that expresses the meaning of its object.
In this section, we will apply the Peircean typology of signs to Roentgen signs in general and to diagnostic signs in particular.
All Roentgen signs that present in radiographic images are sinsigns. Fundamental image qualities such as opacity and opacity gradients or line separation and edge separation of the visual field (cf. Cantor 2000) are representamina of qualisigns that are obtained from sinsigns. All types of Roentgen signs that are recognized by the community of radiologists are legisigns. These include both named and described but unnamed signs.
All Roentgen signs have an iconic character since their representamina are images that are physically determined by their objects. All Roentgen signs also have a symbolic character since the opposition of representamen and object in these signs involves certain conventions. These conventions include the representation of x-ray attenuation by a gray scale, the representation of solid objects by central projections onto a surface (the film) and the representation of objects from conventional points of view (standard projections). However, diagnostic Roentgen signs are distinguished by their indexical character. The pragmatic types of diagnostic signs have been termed detectors, localizers and identifiers (cf. Cantor 2002). A detector is a referential index since detection of an abnormal event involves awareness of the presence of something that is unexpected or the absence of something that is expected. A localizer is an existential index since localization is an entailment of place. An identifier is a nominal index since identification involves meaningful naming. From a logical (categorical) point of view, identification entails localization and localization entails detection. Therefore, an identifier is also a localizer and a detector while a localizer is also a detector. (cf. the categorical inclusion rule in Liszka [1996: 46]).
Roentgen signs involving optical density or a gradient of optical density express attributes of abnormal events and therefore have a rhematic character. In radiography, unexpected optical density represents changed physical density e.g. fluid filling of airspaces, bone loss or soft tissue calcification. In lung, the boundary gradient of a mass density represents its growth pattern e.g. infiltrative or contained growth. In bone, periosteal reaction represents elevation of the periosteal membrane by cells or fluid (transudate, exudate or hemorrhage). All Roentgen signs whose interpretant is a statement about an abnormal event (such as "This is pneumonia", "This is a renal calculus" or "This is a fracture") have a dicentic character. A radiological report is usually an argument (in the Peircean sense) in which diagnoses are inferred from Roentgen signs and clinical information. In some cases, the conclusion of a report is simply the interpretant of a single dicisign.
An entire radiographic image may be thought of as the representamen of a Roentgen sign. The object of such a sign is a region of the human body e.g. chest, abdomen, limb etc. Its immediate interpretant (cf. Houser 1992:xxxvii) is that it represents the gross anatomy of a region of the body as a projection from a particular point of view at a particular time. The mental operation that directs the attention of the interpreter to a part of an image while neglecting the rest yields a subimage (cf. 'prescissive abstraction' in Peirce [1868]: 2). Any subimage obtained in this way may be thought of as the representamen of a Roentgen subsign. The object of such a subsign is an anatomic structure in the region of interest and its interpretant is its meaning. In the next section, we will define semiotic operators that are used to combine and transform Roentgen subsigns in the reading of radiographic images.
In general, a semiotic operator is a process by which signs are produced from other signs. A unary semiotic operator is a correspondence between a single sign and another sign. Roentgen semiosis (the adaptation of Peircean semiosis for Roentgen semiotics cf. Cantor 2000, Figure 11) may be thought of as a unary semiotic operator since it is a process by which a new sign is produced by reinterpretation of a sign. In principle, there is no limit to the number of times this process may be repeated. A binary semiotic operator is a correspondence between a pair of signs and another sign. The new sign produced in this way will be called the resultant of the operator. The signs that are combined by such an operator will be called the components of the resultant. At least three types of semiotic operators are used in the reading of radiographic images. These will be called semiotic union, semiotic product and semiotic action.
Semiotic union
Binary semiotic union is a correspondence between a pair of subsigns of a single image and another subsign of the same image. The representamen of the resultant of this operator consists of the 'copresent' representamina of its components. Two representatamina will be termed copresent if they are both present in the same image. The interpretant of the resultant of binary semiotic union is the sentential conjunction of the interpretants of its components. Iteration of binary semiotic union yields a well-defined ternary operator. In fact, a union of three subsigns that present in a specific order produces a well-defined resultant since binary semiotic union is an associative operator. This means that the union of the resultant of the first and second signs with the third produces the same resultant as the union of the first sign with the resultant of the second and third. This follows from the fact that copresence in the representamen and conjunction in the interpretant are both associative connectives. Clearly, the same argument may be used to define semiotic union for any number of Roentgen signs. Furthermore, the resultant of such an operator is independent of the order of presentation of its components in the mind of the interpreter. This is because copresence and conjunction are both commutative connectives. Therefore, semiotic union is a well-defined polyvalent operator on Roentgen signs.
Compound Roentgen signs
The resultant of a semiotic union of Roentgen signs will be called a compound sign. Compound Roentgen signs may be regional or syndromic.
Regional compound signs. A compound Roentgen sign is a regional sign if its representamen is located within a single anatomic compartment. In the chest, regional compound signs include patterns of lobar collapse (eg. with characteristic orientation and displacement of an opaque edge), a pulmonary nodule (e.g. a round opacity with well defined margins) and pneumonia (e.g. an irregularly shaped opacity with fading margins). In the abdomen, regional compound signs include patterns of small bowel obstruction (e.g. proximally increasing distention of small bowel, arranged in tiers, with a paucity of gas in the colon), patterns of ascites (e.g. bulging flanks, diffuse 'ground glass' opacity, separated or centrally located bowel loops) and patterns of cholelithiasis (e.g. a single round body with layered calcification or multiple facetted calcified bodies in the right upper quadrant of the abdomen). In the skeleton, regional compound signs include patterns of healing fracture (e.g. radiolucent lines in association with new bone formation), patterns of bone infarct (e.g. a zonal pattern of subchondral densities with signs of mechanical failure such as a subchondral crack or collapse of the articular surface) and tumor patterns (e.g. lesions that are osteolytic or osteoblastic, circumscribed or infiltrative, with or without periosteal or trabecular reaction, with or without calcified tumor matrix).
Syndromic compound signs. A compound Roentgen sign is a syndromic sign if the representamina of its components belong to different anatomic compartments. Examples of syndromic signs include patterns of congestive heart failure (e.g. cardiomegaly, pattterns of interstitial or alveolar edema and pleural effusions), patterns of chronic renal failure (e.g. various signs of hyperparathyroid osteopathy, osteomalacia and soft tissue calcinosis) and patterns of rheumatoid arthritis (e.g. cartilage loss and marginal erosions in characteristic distributions).
Semiotic resolution
Semiotic resolution is the mental operation by which a given subsign of a radiographic image is perceived as a semiotic union of other subsigns. It is plausible to expect that the depth of semiotic resolution is limited on clinical images. This means that there are subsigns of radiographic images that cannot be resolved into other subsigns i.e. that are irreducible. The density signs, gradient signs and separation signs described by Cantor (2000) are irreducible detectors. The question remains: Is there a complete set of irreducible Roentgen signs from which all compound Roentgen signs are generated by semiotic union?
Semiotic product
A binary semiotic product is a correspondence between a pair of subsigns from different radiographic images and another sign called its resultant. The subsigns are the components of the resultant. In contrast with semiotic union, the resultant of a semiotic product is not a subsign of a single Roentgen image. It is a sign whose representamen is an opposition in the mind of the interpreter of the representamina of its components. The resultant of a semiotic product of Roentgen signs will be called a composite sign. Composite signs are used for semiotic correlation and comparison.
Semiotic correlation. A composite sign is correlative if its representamen is an opposition in the mind of the interpreter of correlate representamina of subsigns of different images of the same patient. Correlate representamina refer to the same anatomic structure that is represented in different images. The interpretant of a correlative sign is obtained from the interpretants of its components by inference. Hence, semiotic correlation is a binary semiotic operator whose resultant is a correlative composite sign. Although semiotic correlation is a binary operator, it may be applied pair-wise to multiple images.
Semiotic correlation may be synchronic or diachronic. The correlate representamina of a synchronic correlation belong to different images of the same radiographic study e.g. frontal, lateral or oblique projections, where the images of a radiographic study are obtained at approximately the same time. Synchronic correlation may be used for detection, localization or identification of abnormalities. A pulmonary nodule may be detected on a lateral projection of the chest while it is obscured by mediastinal opacities on a frontal projection. A small pneumoperitoneum may be detected on an erect abdominal film (a gain of edge separation sign) while it is not visualized on a supine film. A nondisplaced fracture may be detectable on only one projection of a standard radiographic study by a gain of line separation sign. A detailed description of the separation signs has been given by Cantor (2000). Synchronic correlation of subsigns from orthogonal projections is the standard method for radiographic depth localization. Synchronic correlation is employed in the process of identification when different identifiers of an abnormality present in different images of a radiographic study. The correlate representamina of a diachronic correlation belong to corresponding projections from radiographic studies of the same patient performed at different times i.e. serial studies. Diachronic correlation is a standard method for detecting change or demonstrating invariance i.e. for evaluation of biological behavior or the course of a disease.
Semiotic comparison. A composite Roentgen sign is comparative if its representamen is an opposition in the mind of the interpreter of representamina of subsigns, one from a clinical image and the other from a reference image. The interpretant of a comparative sign is a comparison between the interpretants of its components. Hence, semiotic comparison is a binary semiotic operator whose resultant is a comparative composite sign. Semiotic comparison establishes the identity or differences between signs in the process of being interpreted and signs that have already been interpreted (e.g. sinsigns in an atlas or reference work). Hence, semiotic comparison is used for semiotic identification and discrimination.
Semiotic action
A semiotic action is a correspondence between a Roentgen subsign of an image and a reinterpretation of that sign induced by the presence of another sign, either in the same image or in the mind of the interpreter (e.g. a sign given by clinical history). Hence, semiotic action is a unary semiotic operator. The sign that induces the action will be called an active sign. There are at least three types of semiotic action based on the effect of the active sign.
An active sign that increases the likelihood of an interpretant may be called a promoter. In the chest, bilateral parahilar 'alveolar' infiltrates are suggestive of pulmonary edema and the presence of 'septal lines' makes this interpretant more likely. In the abdomen, small and large bowel distention with an absence of gas in the rectum are suggestive of obstruction while an abrupt termination of distention in the sigmoid colon makes obstructing carcinoma a likely diagnosis. On a lateral projection of the elbow, displacement of the humeral fat pads in the absence of osseous abnormality is a nonspecific sign of the presence of space occupying material within the joint (e.g. transudate, exudate, hemorrhage or hypertrophic synovium). However, a history of recent elbow trauma or tenderness over the radial head makes occult radial head fracture (accompanied by hemarthrosis) the most likely diagnosis.
An active sign that decreases the likelihood of an interpretant may be called a demoter. In the chest, a central calcification within a pulmonary nodule makes a diagnosis of neoplasm less likely. In the abdomen, gas within the rectum in a patient with distended bowel makes intestinal obstruction less likely. In the appendicular skeleton, an irregularity on one side with an identical finding on the contralateral side makes an abnormality less likely.
An active sign that logically excludes (i.e. is inconsistent with) an interpretant may be called an excluder. Examples of excluders are the absence of abnormal pulmonary opacities that radiographically excludes pneumonia, non-distention of gas containing bowel that excludes intestinal obstruction and extension of a radiolucent line outside of a bone that excludes a fracture.
Some Roentgen signs direct the attention of the observer/interpreter toward or away from other signs. Such signs that affect the mind of the observer will be called affectors. Note that affectors are not semiotic operators. Two types of affectors may be termed directors and distractors.
Directors direct the attention of the observer toward specific other signs. In the chest, prominence of the interlobar fissures directs the attention of the observer towards less easily seen septal lines of pulmonary edema. In the abdomen, obvious bowel distention directs the attention of the observer toward patterns of distention that are characteristic of obstruction or ileus. In the skeleton, the presence of periosteal new bone directs the attention of the observer toward the sometimes subtle signs of fatigue fracture or septic bone loss.
Distractors direct the attention of the observer away from other signs in a nonspecific manner. Any abnormal sign that is particularly important or interesting or unexpected may serve as a distractor and contribute to an interrupted or incomplete visual search (cf. Kundel [2000] on visual search).
The naming of a Roentgen sign is a category preserving correspondence between verbal and visual indexical signs. A verbal indexical sign is a triadic relation between word, word-object and meaning. The opposition between word and word-object is in the mind of the interpreter. A visual indexical sign is a triadic relation between image, image-object and interpretant. The opposition between image and image-object is in the physical world. The act of naming establishes a correspondence between categorical types of verbal indexical signs and corresponding categorical types of visual indexical signs. From Figure 1, the categorical types of indexical opposition may be characterized as commonality, entailment and convention. These three types of opposition correspond to three categorical types of naming that may be termed metaphoric, eponymic and acronymic. A diagram of this naming process is given in Figure 2.
Metaphoric naming is based on commonality of appearance, structure or meaning. Names based on commonality of appearance include the 'air bronchogram' sign of alveolar pulmonary disease, the 'ground glass' sign of ascites and the 'ivory vertebra' sign of neoplastic skeletal disease. Names based on commonality of structure include the 'butterfly' pattern of pulmonary edema, the 'string of beads' sign of small bowel obstruction and the 'rugger-jersey spine' of hyperparathyroidism. Names based on commonality of meaning include the 'extrapleural' sign of extrapleural mass, the 'psoas' sign of retroperitoneal pathology and the 'ring' sign of rotatory subluxation of the scaphoid. Eponymic naming is based on the conventionally accepted author of a sign. Names based on this type of entailment include 'Kerley lines' of interstitial pulmonary edema, 'Rigler's sign' of pneumoperitoneum and 'Osgood-Schlatter Lesion. ' Acronymic naming is based on convention and is often applied to syndromic signs. Examples of such names are VACTERL (Vertebral-Anal-Cardiac-Tracheo-Esophageal-Renal-Limb) syndrome, DISH (Diffuse Idiopathic Skeletal Hyperostosis) and CPPD (Calcium Pyrophosphate Dihydrate) crystal deposition disease.
Just as the reading of printed language begins with visual scanning, the reading of radiographic images begins with a visual search (cf. Kundel [2000]). Visual search is the poorly understood process by which the attention of the film reader is directed systematically towards abnormal subsigns of an image. In this way, the reader mentally separates abnormal subsigns from a profusion of normal subsigns. The abnormal subsigns are then combined or transformed in the mind of the reader by means of semiotic operators to obtain (ideally) a single sign the interpretant of which is called the diagnosis. Frequently, a specific diagnosis cannot be obtained from immediately available signs. In such cases, the image reader formulates a set of possible diagnoses with graded degrees of likelihood. This process is called differential diagnosis.
In this paper, we have attempted to delineate a common semiotic basis for the reading of visual images and printed language. Specifically, we have identified a morphology and syntax for Roentgen semiotic grammar. The semiomorphology is based on Charles S. Peirce's categorical typology of triadic signs. The semiosyntax is based on the use of semiotic operators defined on Roentgen signs.