parseltongue/ Intro White Rabbit.Pg

Follow the White Rabbit

No one could ever be told the answer to that question. They have to see it to believe it.

The Problem

You'd be surprised how illusory intelligence becomes once it needs to be proven explicitly.

Large language models don't know what truth is.

Despite the strong flavour of philosophical complaint, I mean this as an engineering fact. Even after processing the internet several times over, and passing training on pleasing humans, solving math puzzles, and coding purple landing pages - LLM agents still have no sense of truth.

What do I mean by "sense"? An absurd example: tomorrow everyone around you agrees that plastic vegetables make a good breakfast. It doesn't matter how many times they tell you, or even eat a piece to prove their point - you won't believe it. That's not the case for LLMs. You can call this human quality being grounded, or reasonable - the terminology doesn't matter much, as long as you get the sense of the problem.

Even when an LLM solves a math puzzle or a coding task, it reasons within the space of what it was trained on. And the bulk of that training was parroting the internet - which, I think we can all agree, is about as poor a source of truth as there can be, with a few notable exceptions.

The problem isn't that agents are stupid - it's that they are becoming very capable at exactly the tasks we train them for. Those include giving us pleasant answers that sometimes happen to be technically correct. This is great if you need a good conversation or a new website. It fundamentally doesn't work in high-stakes scenarios. We don't hire managers who excel at political bootlicking for a surgeon position - sometimes we simply need a different set of skills.

Parseltongue[1] is focusing on addressing the issue of making confident but entirely ungrounded statements. Here is what I mean:

1parseltongue
pltg The Origin ▸
parseltongue = Parseltonguethe-next-sentence-is-grounded-in-parseltongue = true

This sentence is grounded in Parseltongue[2]. And a few things happened: it became represented in the engine powering this documentation[3]; and a fact representing this sentence was created. If you click the reference or the pill, you can locate it in the knowledge graph of this notebook, and in the layer of computations it will participate in next.

2the-next-sentence-is-grounded-in-parseltongue 3sentence-became-part-of-engine
pltg First steps ▸
sentence-became-part-of-engine = truecount-facts-about-sentence = 2

If we count the facts about this sentence that we've introduced so far, we would probably all agree that there are 2[4] of them. This number was calculated by the derivation in the previous block, and if we changed the block - it would update.

4count-facts-about-sentence

What does all this have to do with LLMs lying? So far we were trying to express only the truth. Something interesting happens if we try to represent a lie - for example that there will be cake.[5✖] As we can see, there is no cake - The cake is a lie.[6] But something interesting happens if we try to use those facts in our derivations: we normally calculate 3[7] truthful facts, but if we account for 4[8⚠] - a lie becomes visible! The original lie taints the derivation - even though we can still see its result, we know that there was a lie in its basis.

5there-will-be-cake ✖ 6the-cake-is-a-lie 7count-facts-till-now 8accounting-for-a-lie ⚠
pltg The Fall ▸
the-cake-is-a-lie = The cake is a liecount-facts-till-now = 3there-will-be-cake = there will be cake ✖accounting-for-a-lie = 4 ⚠count-non-lie-so-far = 4

This feature of Parseltongue - catching the propagation of taints - is core to its ability to highlight hallucinations in LLM outputs.

What Parseltongue Does

So what is it exactly? Parseltongue[1] is a formal language for knowledge representation.[9] There are 4[10] primary kinds of knowledge nodes:

1parseltongue 9what-is-parseltongue 10node-kinds
  • A term is a named value - like Parseltongue[1] itself, defined in the very first block. Terms give names to things so the system can reference them.
    1parseltongue
  • A fact is a claim with cited evidence - a quote from a real document that grounds it.
  • A derivation is a computation over other nodes - like the count we calculated above.
  • An axiom is a rule about how other things relate to each other.

This document you're reading is written in .pgmd[11] - a format that wires parseltongue blocks through ordinary prose.[12] It was designed to be a human interface with .pltg[13] - the language of the Parseltongue systems themselves[14][1]

11pgmd 12what-is-pgmd 13pltg 14what-is-pltg 1parseltongue
pltg Observing the Garden ▸
pgmd = .pgmdpltg = .pltgwhat-is-parseltongue = a formal language for knowledge representationwhat-is-pgmd = a format that wires parseltongue blocks through ordinary prosewhat-is-pltg = the language of the Parseltongue systems themselvesverbosity-is-a-feature = verbosity is a featurellm-use-design = designed to use with the LLMsnode-kinds = 4count-facts-garden-block = 8

Notice how the values in the previous paragraph weren't typed - they were resolved from the block above. The system knows where each one came from, what quote grounds it, and whether that quote still exists in the source. If I change the quote - the reference breaks, and you see it.

.pltg[13] is quite verbose compared to typical programming languages, and that verbosity is a feature.[15] It lets us trace facts and relationships with very high precision. Parseltongue[1] was designed to use with the LLMs[16] - for them, writing .pltg[13] modules and documenting things in a somewhat redundant manner is not a problem, and readability is handled by .pgmd.[11]

13pltg 15verbosity-is-a-feature 1parseltongue 16llm-use-design 11pgmd

Seeing It

The answer is out there. It's looking for you and it will find you, if you want it to.

When this document is rendered, it produces four views of the same underlying structure:[17]

17views-share-data
  • Notebook - the prose you're reading, with footnote markers linking to nodes
  • Cards - one card per node, showing kind, value, and verification status
  • Layers - computation-based layout: it shows the path from the core axioms used everywhere to final consequences through each derivation step
  • Graph - the full dependency network, interactive, with taint propagation visible
pltg Knowing Good and Evil ▸
silence = SILENCEconsistency = Consistentgrounding = Groundedtaints = Taintedderivations = Derivedview-count = 4views-share-data = four views of the same underlying structurecount-facts-knowing-block = 7

Those 4[18] views ensure the transparency of the structure. They are different perspectives designed to show how conclusions were reached. The Graph makes it visible what depends on what. The Layers show how results were derived. The Cards and Sources show what was available. The Notebook shows what was reported.

18view-count

This doesn't mean the notebook contains truth or lies in the body of text - the text is still just human text. The difference is that prose sentences can refer to Parseltongue sentences - which are designed to explicitly show whether they are Grounded,[19]Consistent,[20] and honestly Derived.[21] Otherwise, the sentence will show its Tainted[22] or return SILENCE.[23]

19grounding 20consistency 21derivations 22taints 23silence

Since all knowledge in Parseltongue[1] is explicit, we can check whether it follows the rules we might like it to follow:

1parseltongue
pltg Promises and Deception ▸
concepts-not-more = 25expect-concepts = 23intro-is-substantial-but-focused = (< ?n concepts-not-more)readme-expected-focused = truediff-ops = Diffreadme-expectations = true = truethis-block-sum = 4facts-and-terms-so-far = 23focus-check = truereadme-counts = 23 = 23readme-expectations = true = trueungrounded-expectations = 25 ≠ 23

The block above computes that we have introduced 23[24] concepts so far. We could set a goal for a focused intro - say, no more than 25[25] concepts. For now this intro stays focused[26]. This is the result of evaluating our rule: (< ?n concepts-not-more).[27] The code after this check uses the Diff[28] operation. If this document grew too sprawling, the coherence[29] check would fail, much like our main 23 = 23[30] expectation. If the promise is inaccurate - say we try to replace the concept limit with the actual count - the diff will show it: 25 ≠ 23[31]

24facts-and-terms-so-far 25concepts-not-more 26focus-check 27intro-is-substantial-but-focused 28diff-ops 29readme-expectations 30readme-counts 31ungrounded-expectations

The Construct

This will feel a little weird.

This document is part of Construct - the place where both agents and humans learn Parseltongue.[1] It has two modes, mirrored:

1parseltongue
  • Scripts - agent instructions. An LLM loads a script and gains operational knowledge - these are available in the repository and via the provided CLI with pg learn ..topic.., e.g. pg learn kung-fu
  • Scenarios - human side. They focus on building understanding of what Parseltongue[1] is, and how to work with it and keep LLM agents accountable during collaborative sessions
    1parseltongue

Construct contains progressive explanations of the system context - we don't want you thrown down the rabbit hole straight into our Mad Tea Party. Instead we recommend a progressive dive with your favourite LLM assistant.

Name Description Script Scenario
white-rabbit Introduction - what Parseltongue is and why it exists -[32] ✓ ready[33]
kung-fu Bench mastery - inspection, search, lens, diagnosis ✓ ready[34] -[35]
to-connect pgmd notebooks - prose wired through with truth ✓ ready[36] -[37]
dodge-bullets Screening, diagnostics, consistency -[38] -[39]
jump-program Resolving dynamic refs, building consistent graphs -[40] -[41]
no-spoon-bending Effects, verify_manual, bending accepted terms -[42] -[43]
read-the-code Grounding layer - documents and data to facts and axioms -[44] -[45]
about-matrix Systems, composition, fundamental lang -[46] -[47]
the-truth Epistemics, std lib, grounding module, diffs -[48] -[49]
to-exit Scoping, projection, delegates -[50] -[51]
to-fly Graph navigation, search, cross-navigation -[52] -[53]
32script-white-rabbit 33scenario-white-rabbit 34script-kung-fu 35scenario-kung-fu 36script-to-connect 37scenario-to-connect 38script-dodge-bullets 39scenario-dodge-bullets 40script-jump-program 41scenario-jump-program 42script-no-spoon-bending 43scenario-no-spoon-bending 44script-read-the-code 45scenario-read-the-code 46script-about-matrix 47scenario-about-matrix 48script-the-truth 49scenario-the-truth 50script-to-exit 51scenario-to-exit 52script-to-fly 53scenario-to-fly

2[54] scripts and 1[55] scenario are live. Right now the agents can already learn kung-fu and how to-connect it to .pgmd[11] notebooks. The explanation for you about how to steer them is WIP.

54scripts-ready 55scenarios-ready 11pgmd

So, if you haven't yet, tell your agent to:

pip install parseltongue-dsl
pg learn kung-fu
pg learn to-connect

And enjoy the show of an LLM bumping into Parseltongue guardrails. You'd be surprised how illusory intelligence becomes once it needs to be proven explicitly.

pltg Grounding ▸
ready = ✓ readytodo = -script-white-rabbit = -scenario-white-rabbit = ✓ readyscript-kung-fu = ✓ readyscenario-kung-fu = -script-to-connect = ✓ readyscenario-to-connect = -script-dodge-bullets = -scenario-dodge-bullets = -script-jump-program = -scenario-jump-program = -script-no-spoon-bending = -scenario-no-spoon-bending = -script-read-the-code = -scenario-read-the-code = -script-about-matrix = -scenario-about-matrix = -script-the-truth = -scenario-the-truth = -script-to-exit = -scenario-to-exit = -script-to-fly = -scenario-to-fly = -scripts-ready = 2scenarios-ready = 1

What's Next

we're all mad here

Unfortunately, there are many more kinds of deception than outright lies, incoherent statements, or ungrounded hallucinations. Agents can still stretch perfectly grounded facts, provide incomplete answers, or hide the truth in other ways - not intentionally, but simply because they don't know how not to.

As with any language, using Parseltongue[1] accurately requires mastery. And mastery comes from deeper understanding.

1parseltongue

Understanding follows curiosity, and if - unlike Alice - you haven't yet followed the trails leading into the web of Parseltongue's[1] Wonderland, I highly recommend you pursue that option.

1parseltongue

Here are two statements: the true = true[29] readme expectations check, and the alternate reality 25 ≠ 23[31] of the ungrounded diff. Take any pill - this is your fairly classical choice, and understanding why the red one is red leads much deeper.

29readme-expectations 31ungrounded-expectations

But if you want to join the Mad Tea Party without being constrained by binary choices, we have another pill:

23 = 23[30]

30readme-counts

And the most interesting question in this notebook is: "Why is it yellow?"

[1]parseltongue = Parseltongue
[2]the-next-sentence-is-grounded-in-parseltongue = true
[3]sentence-became-part-of-engine = true
[4]count-facts-about-sentence = 2
[5]there-will-be-cake = there will be cake
[6]the-cake-is-a-lie = The cake is a lie
[7]count-facts-till-now = 3
[8]accounting-for-a-lie = 4
[9]what-is-parseltongue = a formal language for knowledge representation
[10]node-kinds = 4
[11]pgmd = .pgmd
[12]what-is-pgmd = a format that wires parseltongue blocks through ordinary prose
[13]pltg = .pltg
[14]what-is-pltg = the language of the Parseltongue systems themselves
[15]verbosity-is-a-feature = verbosity is a feature
[16]llm-use-design = designed to use with the LLMs
[17]views-share-data = four views of the same underlying structure
[18]view-count = 4
[19]grounding = Grounded
[20]consistency = Consistent
[21]derivations = Derived
[22]taints = Tainted
[23]silence = SILENCE
[24]facts-and-terms-so-far = 23
[25]concepts-not-more = 25
[26]focus-check = true
[27]intro-is-substantial-but-focused = (< ?n concepts-not-more)
[28]diff-ops = Diff
[29]readme-expectations = true = true
[30]readme-counts = 23 = 23
[31]ungrounded-expectations = 25 ≠ 23
[32]script-white-rabbit = -
[33]scenario-white-rabbit = ✓ ready
[34]script-kung-fu = ✓ ready
[35]scenario-kung-fu = -
[36]script-to-connect = ✓ ready
[37]scenario-to-connect = -
[38]script-dodge-bullets = -
[39]scenario-dodge-bullets = -
[40]script-jump-program = -
[41]scenario-jump-program = -
[42]script-no-spoon-bending = -
[43]scenario-no-spoon-bending = -
[44]script-read-the-code = -
[45]scenario-read-the-code = -
[46]script-about-matrix = -
[47]scenario-about-matrix = -
[48]script-the-truth = -
[49]scenario-the-truth = -
[50]script-to-exit = -
[51]scenario-to-exit = -
[52]script-to-fly = -
[53]scenario-to-fly = -
[54]scripts-ready = 2
[55]scenarios-ready = 1
Diagnostics6 errors · 101 warnings · 68 info
error 6 ▸
warning 101 ▸
info 68 ▸
System 35 facts 55 terms 61 axioms 21 theorems 175 issues

?
Developed by sci2sci
Need to convert data and documents to knowledge safely at enterprise scale?
Try VectorCat!