The Santa Fe Institute offers a brilliant learning portal by the name Complexity Explorer which offers some really great and mostly free courses on complex dynamics and its applications in various domains.
I started learning from this particular course on Humanities Analytics by lecturers David Kinney & Simon DeDeo and its characteristics to understand how complex systems science finds its footing in social science and humanities research.
DISCLAIMER: The following are my PERSONAL NOTES from what I learnt from this course, it holds my personal copyright, and not in any way an attempt to plagiarize the main course offering by Complexity Explorer. Here I just present the lessons I learnt, in my own way, in my own language, and should not be considered as an official coursework.
CASE STUDIES
Learning Goals:
1) understand correlations among words and patterns of words and how it can support scholarly arguments
2) compare correlative features in different studies and the reason behind selecting these specific features In the assignment
3) propose applications for these correlations to your own work to support (or refute) any argument of choice.
Important Definitions
SIGNAL: A fact X is a signal of a fact Y, i.e. knowing X tells us something about or reduces our uncertainty about, Y. This usage contrasts a little with the standard use, because a signal often indicates some kind of intentionality (X is about Y), or agency (a person uses X deliberately to inform you about Y), or causality (X signals Y only if, for example, X precedes Y in time).
Below are some examples to highlight the usage of signals.
EXAMPLE 1:
Zip code is a signal of income: If zip code is a signal of income it means that “in general” knowledge of zip code helps improve the accuracy of the knowledge or idea (more likely, believes) about income, although not precise or perfect information or belief can be gathered from it. Knowing the zip or postal code that a person lives in, one can gain some idea or information about his/her income. Although not precisely, but it will lead to refine beliefs about the income. eg, if the zip code is associated with (say) Greenwich, Connecticut (a posh part of East Coast, USA), one would consider it more likely than usual that the person’s income is quite high.
Signals can be imperfect and also not precise.
An imperfect signal may only incomplete or minor information, may work great sometimes, but might not at all.
All SIGNALS are generally symmetric in behavior, hence,
EXAMPLE 2:
Income is a signal of zip code: similarly, if knowledge of X tells about Y, then knowledge of Y tells about X. Knowing that someone’s income is high, by this relation, will tell if they’re more likely to live in one of a small number of zip codes located in some fancy parts of major coastal cities, in general costly areas to live in.
In information theory, this symmetry is observed precisely. The strength of the signal that X gives for Y is precisely equal to the strength of the signal that Y gives for X.
Prediction: X predicts Y when X is a signal of Y. Prediction can be retrodiction, meaning that we might say that X predicts Y even when X comes after Y. In general, we talk about prediction from the point of view of an omniscient observer.
PREDICTION: X predicts Y when X is a signal of Y. Prediction can be retrodiction, i.e. X predicts Y even if X comes after Y. In general, we talk about prediction from the point of view of a 3rd party all-knowing observer.
Correlation A correlation is a relationship between two variables or quantities such that the value of one is a signal of the value of the other.
Correlation often is used to mean linear correlation, both units decrease or increase in a straight line (linearly) with a change in the other.
A positive correlation means that one quantity increase with the increase in other, or decrease with the decrease in the other. Negative correlation, on the other hand, means one quantity decrease with the increase in other, or increase with the decrease in other.
Signal, correlation, and prediction are almost synonymous terms, but are used in different contexts.
Signals are used when there are many different features of a system, and one is interested in the relationships. Correlations are used, when these signals are quantitative. And prediction is used when we’re interested in interpreting how different signals combine to tell us about our chosen question or topic.
Eg. income, exam outcome, postal code, and race are signals of each other. But we might be want to know how “income, race, and zip code” combine to help us predict performance on an exam.
Information-theoretic: Pioneering work by Claude Shannon (at AT&T Bell Labs), together with insights from wartime encryption tasks, developed a new science of signals, patterns, and prediction, called Information Theory. An information-theoretic account of a record, then, is discovering patterns and signals, and then understanding the underlying events in terms of how those patterns and signals interact with each other.
Operationalization: Turning ideas into measurable quantities. In any study, we take a difficult, cultural and complex idea, and interpret some aspect of it in a quantitative form.
EXAMPLE:
Before advent of computers, one used to operationalize membership in a particular social class by tracking visible markers like income, car used, consumption of media.
What it meant to be a part or member of a particular class is complex and subject to interpretations; but tracking number of times a person goes to the opera is not. The latter can be counted easily, and facts about those numbers may highlight facts about the deeper ideas. Like, counting opera-going times might be used to measure how immigrants moved up the social class ladder across generations.
A good operationalization does not redefine the concept of interest, rather it makes an argument for why the idea, as best understood, may lead to certain measurable consequences, and why those measurements might provide a signal of the underlying concept or idea.
Operationalization can get pretty theoretical. Eg. In natural language processing, a common technique is to operationalize certain semantic concepts like synonyms in terms of syntactic structure (i.e. two words that tend to occur nearby in a sentence are more likely to be synonyms).
A good operationalization can provide completely new perspectives in long-standing debates. Conversely, bad operationalization can mislead and misrepresent. Like how the Western culture operationalized “intelligence” as performance on a geometric reasoning test.
Hermeneutic circle: Traditionally, the hermeneutic circle refers to the way in which we understand some part of a text in terms of our predominant ideas about its overall structure and meaning — but that we also refine or update our beliefs about it in response to particular moments within the text.
Something similar happens with the process of operationalization. One can operationalize an idea by selecting some part of it – often small and probabilistic feature – and seeing what can be learnt from it.
In a good investigation, however, our learnings from looking at this quantified characteristic then influences how we understand the concept overall.
For example, one might learn that a previous operationalization, which seemed justified at the beginning, was only because we didn’t understand the concept well enough. The quantitative results might deepen our understanding of the concept – and, in turn, suggest new ways to refine and improve the operationalization.
3 Examples to highlight studies in Cultural Analytics
1. PROCEEDINGS OF THE OLD BAILEY
The digitization of the PROCEEDINGS held in OLD BAILEY, London’s Central Criminal Court from 1674 to 1913 culminated in the creation of this record https://www.oldbaileyonline.org/index.jsp
“This is a one of the largest bodies of texts detailing the lives of non-elite people ever published, containing 197,745 criminal trials held at London’s central criminal court.”
“The Old Bailey Proceedings Online makes available a fully searchable, digitised collection of all surviving editions of the Old Bailey Proceedings from 1674 to 1913, and of the Ordinary of Newgate’s Accounts between 1676 and 1772. It allows access to over 197,000 trials and biographical details of approximately 2,500 men and women executed at Tyburn, free of charge for non-commercial use.“
The site is hosted by The Digital Humanities Institute.
This example focuses on the how the Official Recorded INDICTMENTS (formal charge or accusation) made post an accusation of crime, and the TESTIMONIES of the involved parties correlate to each other.
For example for a CASE OF A THEFT from a person named X by two people YZ:
- There is one official indictment that accuses YZ of stealing from X
- There is also X’s testimony of what transpired the fateful night, how he met YZ, what dialogues were exchanged and so on.
HOW DOES THIS NARRATIVE (testimony) PREDICT the INDICTMENT is what we can try to understand from the entire record of the event.
This is what (mostly) HUMANITIES ANALYTICS does: Ask about the relationship about the relationship between the formal indictment (or recorded version of an event) and the dramatic human retelling of the moment (narratives of the parties involved) i.e. we aim to build the context or rather connect it to the various subtexts.
We can look at this CASE OF A THEFT as a classic example of the SIGNAL SYSTEM
The SIGNAL SYSTEM correlates:
a) What does one thing (trial as an event): What is said during the trial (testimonies, witness accounts, questions, etc.)
b) Tell about another thing (trial as part of state’s legal mechanism): What the trial is actually for (how the state describes the issue)
Based on the above system, we can now plugin the machine i.e. develop and use computer algorithms to develop a BINARY SYSTEM (distinguishing VIOLENT and NON-VIOLENT crimes), to measure: how court transcript (written records of event) predicts the indictment (formal charge sheet, distinguishing between violent and nonviolent crimes)
Based on this BINARY SYSTEM now we can plot a graph (Shown below) on the SIGNAL STRENGTH i.e. how strongly Indictment predicts Transcript of the event.
This graph is an attempt at tracking long timescale cultural shifts of how the state has come to control crimes as a state machinery. This is invisible to the eyes at the first however as we proceed through the various recorded trials through the 100 years, we see visibly how the state has come to control, track, monitor crimes and indictments of various degrees (both violent and minor non-violent ones).
That is, the evolution of the nation’s and in broader sense, the world’s criminal justice system and legal mechanism are being highlighted.
Here we see as how with the year’s proceeds, our information becomes broader, more objective, distinct and quantifiable enough to distinguish between indictment and transcript in the 100 year analysis, this distinction is visibly absent initially.
Suggested Reading
OPERATIONALIZING: As highlighted above, we operationalize social and cultural issues or systems i.e. turn ideas, issues or concepts into quantifiable units.
2. PROZHITO: 500,000 DIARY RECORDS
Prozhito is a Russian venture of a large scale digital archive of Diaries from Russia (and Soviet) from 1900s to the 2000s. Around 1700 authors, 4000 diaries and 450,000 entries are stored in this digital archive.
For the context of this example, lets first put forward a research question: “HOW DO PEOPLE PERCEIVE THE ACT OF DIARY-KEEPING OVER THE YEARS?” and try to find a solution to this using Humanities Analytics.
The approach to answer this question is to find and analyze the words, pattern of words used that surround the word “DIARY” (based on sheer physical proximity to the word diary) by going through all the archived records in PROZHITO.
That is, we aim to find how the word diary is used in different contexts and evaluate or rather quantify the retellings based on a case by case evaluation of the diary records, reflections about how the author tend to connect with the term diary. (This is an example of OPERATIONALIZING where we are taking simple diary entries of people and turning it into quantifiable indicators based on perception and reflection, just by physically analyzing how the diary is used in different contexts based on the words that lie adjacent to it in usage as meant by the original authors.)
For this case by case analysis we would now plugin a machine, and train the algorithm to take all these entries and for each of these find common patterns or styles and method of addressing or talking about the idea of DIARY, simply through word usage.
This has been done exactly by a team of experts Sergei et all and based on extensive machine processing, the following broad categories were found to be connected with Diary-Keeping: (the specific naming was done by the team based on looser definitions that resulted based on the analysis)
- SPIRIT: diary seen as a tool for spiritual development.
- ROUTINE: as part of daily routine.
- LITERARY: as a tool for literary work
- MATERIAL FORM: connected with the texture of pages, smell, how the diary’s shape caught the author’s attention, type of ink used.
- INTERPERSONAL: diary for communication, sharing thoughts with another person and so on.
A plot was made to analyze the dynamics of evolution throughout time of these broad categories as a fraction (shown below), interestingly there was no visible difference in the context in which diary writing was being perceived or associated with by diary writer over all these decades.
This is called PATTERN STABILITY.
A similar study was conducted to find how people associate with official manifestos and essays over the years 1900-2000, the result show very drastic changes over the decades as is visible in the below graph.
The example of manifestos portray PATTERN INSTABILITY.
THE ANALYTICS OF FRENCH REVOLUTION
Here we analyze how the French Revolution can be used as a Pattern Machine to study pattern making and pattern breaking.
Here we are concerned of Transcripts of debates and speeches that happened during and/or surrounding the FRENCH REVOLUTION.
PATTERNS can be identified as self repeating designs (words, figures, shapes, ideas, etc.)
Here analyzing the transcripts we find each speech decomposes into a combination of patterns. (More similarly to the Russian diary archives which was used to categorize using patterns of words).
For example if we define a particular set of 100 patterns of said IDEAS and THEORIES that were being propagated or used to motivate people against the royals and aristocrats.
Now to understand what these pattern 5, 22 and 17 are, we need to exemplify some of them here.
100 patterns
Some of these predefined patterns to classify the speech are as follows: (all pics are taken as is from the course website and YouTube video)
Using these PATTERNS, we would like question the PREDICTABILITY OF DEBATE i.e. how do patterns in one said speech predict what patterns would appear in the next speech. Following up on this, we would need to define a term novelty:
NOVELTY: in this context is a speech or dialogue that has significantly different pattern combinations than any of the previous speeches or dialogues i.e. in short, a speech of entirely new (novel) ideas.
Based on the concept of novelty, we can distinguish between the LEADERS AND THE FOLLOWERS
LEADERS: have high novelty, introduce new ideas and patterns, innovate, do not imitate others.
FOLLOWERS: low novelty, imitate speech patterns of previous speakers.
But from this definition, one can also say that an exemplary LEADER (say like Robespierre) can be mistaken with a local lunatic or fool who might also speak out dialogues of novel patterns, even if in ignorance of context or sheer tomfoolery.
Hence, novelty cannot be the sole deciding factor. We therefore need a more refined definition, which is:
RESONANCE: a rhetorical tool or power, based on a combination of much one innovates (novelty) and how much the innovation is imitated by others.
This definition is suitable because a LEADER like Robespierre will be more resonated or rather imitated by other speakers, however the local lunatic although delivering unique speeches wont be imitated by others.
This brings us to the Four Part Classification
Four part classification
1. High novelty, High Innovation: speaker who introduces new ideas that sticks around. (eg. Robespierre, Alexandre Lameth, Charles Antoine Chasset, Villeneuve)
2. High Novelty, Low Innovation: speaker who introduces new ideas but is majorly ignored by others
3. Low Novelty, High Imitation: this is an ideal follower as this person keeps the discussion on track
4. Low Novelty, Low Imitation: speaker who is either uninterested or is ignored when he tries to keep the discussion on track
This culminates our 3 examples of Humanities Analytics in Practice.
Quick Recap:
Signals: what does one thing predicts about another
Patterns: self repeating designs (habits and practices), how we put things together.
Feminism in Data Science
For the next section in this coursework: follow this link which opens another similar summarized note essay by me: DATA FEMINISM
Analyzing Excellence in Humanities
This section tries to understand how a thick research question (here, “What is excellence in the humanities?”) can be addressed by analyzing the characteristics like word patterns in specific corpus, like reviews for a book (blurbs).
Some important terminologies that might come handy:
1. Blurb: A short quote or anecdote, a small appreciative comment by someone (other than the author) on the back of a book. In the academia, blurbs are provided normally by fellow academics, or by newspapers. Blurbs be a single word to an extensive summary.
2. Rhetoric of excellence (encomium): In general, rhetoric refers to speech (or writing) in a social context; a more distinct definition is persuasive speech (or writing).
The study of Rhetorical devices, or figures of speech, i.e., a pattern of word usage can be found in various semantic contexts.
eg. sarcasm, a way of using words (mostly used in humor to convey irony in order to mock) that could have various interpretations and meanings. zeugma is the use of the same word in two different senses in the same sentence like ‘he caught a fish, and a cold‘.
The rhetoric of excellence is an attempt at defining questions that blurb study can help answer. Blurbs are full of figures of speech, as well as more domain-specific devices — e.g. how blurbs on the back of philosophy books may appear to appreciate or praise the book in part on how difficult it was to comprehend.
3. Illumination: A term where metaphors associated with light are associated with the author or the book. eg. the author sheds new light on an old problem, or the author provides a new set of lenses through which to view the French Revolution.
Simon Dedeo (the course lecturer) made this up by looking at patterns. In a real analysis one can see how pattern analysis can surface unexpected patterns and rhetorical tropes.
4. Paratextual genre: In literary sense ‘Paratext is material that surrounds a published main text (e.g., the story, non-fiction description, poems, etc.) supplied by the authors, editors, printers, and publishers’, as per Wikipedia. ‘The additional materials form a framework for the main text, and can change reception or interpretation of a text’. Blurbs are part of paratext.
An important question arises that is ‘why use complicated words for something easy to describe?’ Simon feels that these terms, albeit feels heavy initially, can serve useful taxonomic purpose, and help us to sight connections we might miss otherwise. eg. if we’re studying the paratext of a published book, there will be similarities and differences (what?) than the paratext of an online fan fiction.
Paratexts also help in connecting different time periods; in the 1850s, eg. books were often sold loose-leaf and bound by a bookbinder’s after sale — so talking about comments on the back cover are useless. If the jargon points to a coherent phenomenon, it can be very useful.
Signal and Sign Here the word signal is used in a different sense from the rest of this essay. Mainly because scientists often differ in word usage from more conventional uses. For here;
5. Signal (informal): Using symbols or language to communicate a message. Example, a traffic signal is set to Green to show it is safe for drivers move, or how a recommender might write ‘Jack is an excellent scholar of the humanities’ to mean as is intended.
6. Sign (informal): Signal but in the informal sense. A sign is something that provide information about something else, whether or not it is intended. Example, a sign that it is safe to drive through an intersection might be the absence of noises from other cars. A recommendation letter that repeatedly misspells the person’s name is giving a sign that the recommender perhaps doesn’t know the person well.
7. Consilience: A consilient theory is one that brings in different observations together under the same conceptual umbrella. These observations may well be perceived as distinct and unrelated, however a consilient theory shows how they may be related under the same description and explained together. Classic examples include the connection of gravity of earth to the patterns of planetary motion in the solar system; the connection between electricity and magnetism in the study of electromagnetism; or the “New Synthesis” of Darwinian evolution. It can read in details in this paper here.
The pursuit of the question of excellence in humanities requires it to be viewed as a cultural practice and see ways how participants in it intentionally send signs and signals of what they believe to be excellence. Signs and Signals here are meant in the informal way as described above.
Benjamin Franklin’s Kite Experiment, a so called life threatening death defying experiment to demonstrate the electrical nature of lightning (experts claim Franklin’s kite actually picked up some ambient charges as had lightning hit the kite he would have been electrocuted), is often described as a Decisive Experiment in proving or disproving the theory of lightning be electrical charges. This is an example of consilience where we are concerned with the much talked about laws of the universe.
The book A New History of Humanities by Rens Bod discusses crucial aspects of the difference between the natural sciences and the social sciences. Humanities as a field is far more flexible and accommodating towards criticisms, different interpretations, differential views and opinions. However in the natural sciences, is theory is held valid only till it is failed by experiment or refuted by new research, there is no space for contrasting views to exist simultaneously. Summarizing two main differences are:
1) Competing Interpretations as explained above.
2) Novelty or exceptionality as in what is new about the ideas of the book, is it seminal paradigm shifting work, even with broad generalizations for example the world war I and II although both being similar war between nations was unique and different in every aspect to each other, and so was the suffrage and abolitionist movements.
Get the book here or,
Seeing such concepts from the lens of Humanities Analytics would help shed more light into the granularities of the topic.
Empirical Study of Excellence in Humanities
Studying or analysing blurbs behind a very popular book Stuff and Money in the time of the French Revolution by Rebecca L. Spang gives us a short description of the qualitative aspects of the material and writing of the book. An analysis of what each keyword from the blurb is a reflection of can be understood by the following screenshot taken from Simon’s presentation.
Buy the book here
The sociological notion to what is Rhetoric in Excellence might be on how institutions form around ideas of merit? From a historical aspect might be how and when a certain paratextual genre emerged and what does it teaches us about documents from historical periods of significance. However Humanities Analytics would be just getting hold of an extensive data set of blurbs and analysing it for patterns.
Patterns in rhetoric of excellence
The above picture highlights certain patterns that emerged while studying the blurbs for the book Thinking and Being by Irad Kimhi.
Get the book here or,
The blurbs can be distinguished into distinct patterns which tells the following about the book:
A) the blue highlighted words indicate the book is a difficult to read and comprehend easily, unlike Spang’s book in the previous section.
B) The red words however goes onto say that inspite of being a difficult read this book is a pleasure and gratifying to those who could comprehend the material.
C) The green words suggest that the work and ideas described in the book are paradigm-shifting, high novelty, seminal.
This is how analysing blurbs or rather the short commentaries/reviews on a book can tell us about the book in a larger qualitative sense.
Asking the Right Questions
Questions are fundamental in keeping the research on track. Raising quality questions for Humanities Analytics research is important, beginning with a broad question and working towards a narrow one, which may again become relevant to a second question that this even broader than the initially considered question.
A good research question irregardless of its answer, draw’s a reader’s attention to novel details in the domain of inquiry. Most good questions primarily concern ideas than evidences. Following are few important terms.
Research Question: A concise question statement, (ending with a question mark) that the research intends to answer, ‘which helps organize the investigations, directs one attention and keep everyone on board‘. Jargon-free and easily understandable for laypeople are two important characteristics for a good question. A research question can often vary with differences in authorship, methodology and resources used, even if within the same narrow domain. Research questions also evolve with time and depth of study or findings.
Sociology of Knowledge: The study of the social dynamics occurring in the research process. This includes, eg. study of how scientists collaborate, or choose research questions worth pondering upon.
Empowering and Advancing a Field of Inquiry Good research studies are motivation for both the domain, the researchers and the intended audience. Other than providing answers these also provide a base for new research questions. “In some cases, an investigation can empower others by presenting a new archive or analysis method; in other cases, it might reveal new puzzles in the field, or show how a classic puzzle can be reframed, enabling new questions.“
Computerizing a Field of Inquiry: “Our course is about the ways in which STEM concepts can be used to help phrase and answer new questions in the humanities, or to provide a complementary perspective on classic questions“. Here computers come in to provide an unique perspective and/or analytical tools to probe into a new or existing research problem, often in an attempt to conform to already accepted results.
Few questions that can be raised from the context of our previous example on book blurbs, could be “what do academics value?”, what is the origin of academic blurbs?, “what are the genres of praise?”
How do Research Questions evolve
The following picture highlights as what the authors intended to describe as the eventual evolution of a thick or broad question into a narrower one, which may again lead to newer broad questions.
Often the question we enter with is not the one we leave with in academia. How one genre leads to another genre through the depth of these questions can be illustrated by the following example. Here we can see how a generic question related to discovery led to a potential research question in the genre of enlightenment.
Good vs Bad Questions
A GOOD question tends to be intriguing regardless of its answer, and should connect well with the broader ideas of the field, i.e. be useful beyond its narrow domain. A good question narrows down, put contributors on the same page and also often focusses on something new. A good question also focusses more on the underlying idea and not on the evidence (methodology, costs, etc.) first, and advances and develops the field forward.
A BAD question will be one that is directionless (unfocussed) and terminating in nature, i.e. it will terminate in a dead ended answer. The example of a bad question that Simon (instructor) puts out is:
Can method X do thing Y? This only has two direct approaches for answer. YES it can or NO it cant.
Another example: “What’s up with X?” is a bad question (unfocussed and wide) meanwhile “What’s up with this feature of X?” or “Why/How/When X happen?” these tend to be a bit more directing and focused.
Questions like “This research cost million dollars?” has only one answer “YES” and is a bad question as it focusses on the logistics more than the research.
An algorithm created to detect patterns in Shakespearean plays might recognize King Lear as a Tragedy. Here a question like “X is well known in our field but can a computer detect it?” is a bad question yet which contains the seed of a good one like “What features of King Lear are tragic?” A question depending on solely computerization cannot be called a good one.
Cover Image Source: online-learning.harvard.edu
