Novel Sentiment Analysis

Does a novelist have an emotional shape you could plot? 2019, and the answer was no.

The question

An author's books might share an emotional shape — some arc you could see if you plotted feeling against position in the text. Find it in one Fitzgerald novel, check it against the other three, and you'd have something.

How it works

  1. Read the novel in as one string — This Side of Paradise, off Project Gutenberg.
  2. Split on blank lines. Two newlines is a paragraph break in a Gutenberg text, so this is the unit of analysis, and it is also the thing that killed the project.
  3. Score each paragraph with TextBlob's polarity — −1 for negative, +1 for positive, looked up word by word against a dictionary of general-purpose valence.
  4. Plot polarity against paragraph number. The x-axis is the book, front to back.
main.py — the whole of it
paragraphs = novel_as_string.split("\n\n")

for single_para in paragraphs:
    x += 1
    blob = TextBlob(single_para)
    sentiment_value.append(blob.sentiment.polarity)
    sentiment_position.append(x)

plt.plot(sentiment_position, sentiment_value)

There is no window, no stride, no smoothing and no normalisation. One paragraph in, one number out, straight onto the axis. That is the entire design, and every problem below follows from it.

What came out

The result

The plot is noise. Polarity swings hard between adjacent paragraphs and never settles into anything you could call an arc. No shape to compare between books, so nothing to compare. Shelved a few days after it started.

Being precise about why is the useful part:

  1. A paragraph is the wrong unit. It is short enough that one sarcastic line, or one word of dialogue, flips its sign. That variance swamps whatever slow signal might be underneath it.
  2. Lexicon sentiment does not read fiction. Irony, free indirect style, a beautiful description of something terrible — a dictionary of valence gets all three backwards.
  3. Nothing is smoothed. Even if a trend existed, plotting raw per-paragraph values would bury it.

Where it went

The fixes are windowing and smoothing — score overlapping chunks rather than paragraphs, then take a rolling mean. That is exactly what Prose Similarities does a year later with its scale and jump parameters, and it does produce curves you can read. The question itself was retired properly by Literature Mutations, which drops sentiment altogether in favour of distinctive vocabulary — and finds real structure.

Novel_NLP_Analyzer on GitHub ↗