Where This Idea Came From

I built a skill that pulls highlights and handwriting off my reMarkable tablet and into Google Docs. The highlights were a solvable technical problem. The handwriting was my big blocker, and a personal one. Before any agent could turn my pages into text, it had to learn how I write, and I had to decide where that knowledge would live.

I've been drawing for as long as I can remember. I minored in studio art in college, and these days I handletter, paint, and draw in my down time. A habit that follows me into my notes anytime I have a writing utensil in my hand. A page about how the brain processes emotional or physical threats might open with a looping brush script, drop into all-caps print, throw zigzag flourishes around a heading, and end in a boxed diagram with chunky arrows pointing at the word that matters. My notes often look more like a creative project than an academic outline.

‍

Problem

One page of mine can hold four hands: block capitals, brush lettering, fast cursive, and a compressed scrawl I use to squeeze a note between printed lines. I switch between pencil, calligraphy pens, brushes, and fine points. When a margin fills up, I write sideways up the edge of the page and draw an arrow to connect the pieces.

My marks carry meaning too. A line under a phrase might be emphasis, a divider between two notes, or the edge of a box. A deliberate strikeout means "leave this out." Some words I can't even read myself.

The first problem is figuring out how to teach that to a machine or agent. Then I only want to have to do it once, so where does that knowledge live? A detailed record of how my handwriting works, plus the pages it was built from, feels private. I wanted to decide who gets access to it. So I passed on community tools that would have sent my page images to things like Google's Vision API, and I didn't want the rules sitting inside any one assistant's memory.

‍

Steps to Solve

1. Started with easier examples and corrections, not a rulebook. Claude transcribed four of my documents, 14 pages in all, and I checked every line. Shockingly, it got +90% right. It flagged 11 words it wasn't sure about, and all but one were correct. The biggest error was reading order: it put two margin notes in the wrong sequence.
2. Turned each correction into a rule. The ordering error taught the first one: my arrows set the order I wrote things, so an agent should follow the connector, never the position on the page. Some arrows run from a printed subtitle, like "Does Coregulation Exist in Adulthood?", to my note, which makes the note a question about that section. Every rule went into one guide file.
3. Then Ran a harder round. Round two used four more documents with brush lettering and dense clusters of notes. Claude flagged five words across 14 pages, and its mistakes shifted from reading words to reading marks. It took the edge of a box for a strikethrough. It read an underline as emphasis when I'd drawn it to separate two notes. It reported the detached dot of my question mark as a mark of its own. Each of those became a rule, along with one I like best: Sometimes I can't read a word either, if it’s ineligible, drop it instead of guessing. I will figure it out.
4. Set a rendering rule. Thin arrow tails and box corners disappear at low resolution. The guide now calls for a full page at 170 DPI plus close-up crops at 300 to 400 DPI of anything dense or ambiguous.
5. Kept the guide in Fulcra. The guide sits in my Fulcra account, inside the skill's folder, where I control who reads it when they can access it.
6. Learned from a bad first run. Nobody gets it right the first time, and developers are creators too, so the first iteration is always a draft. One early run started before it found the guide, used a single low-resolution pass, and made the exact reading-order mistake the guide warns about. We threw that Doc out and ran it again. That's the best evidence I have that the guide works.
7. Lastly, I tested it on a notebook. My handwritten notebooks have no printed text underneath, just my thinking on ethics and philosophy, edge to edge. Across four pages, Claude got two words wrong, both in brush lettering:
   • It read one heavy brush-lettered word as crossed out, which would have deleted the word "Autonomy" from my notes. The new rule tells them apart: a real strikeout has looping cancel strokes, and brush lettering keeps its letterforms.
   • It read "human flourishing" as "holistic" and defended the guess by counting ascenders. Brush lettering throws spikes that look like ascenders, so the count proved nothing. The new rule: flag a doubtful word plainly, with no argument attached, and let meaning and my own vocabulary decide. Higher zoom didn't fix either misread. Context did.
   • It also flagged a word as unreadable when I'd written the same phrase in full two inches below. Now it searches my own pages before asking me.
8. Let it fix my spelling. The guide tells Claude to correct my misspellings silently: "resiliancey," "delimma," "embalanced," and "socialtal" all come out right. "REMEMEMBER!!" stays exactly as written, because I meant it, mostly.

‍

Solution

The handwriting guide is a living file in my Fulcra account. Any agent I authorize reads it before transcribing a page, and every correction I make becomes a new rule. It now covers reading order, what each kind of line means, how to handle brush lettering, how to treat notebooks, and where errors tend to cluster.

Keeping it in Fulcra gives me control. The guide lives in my account, not in one assistant's memory. I can edit it, move it, or cut off access in one place. A new agent reads the same rules on day one, so I never retrain anyone. Claude still sees my page images when it transcribes them, and I'm comfortable with that. What I wanted was to decide who holds the full picture of how my handwriting works.

‍

Result

I have notebooks of handwritten notes now transcribed to text documents my agents can mine for insights and thinking, in my own words. I was shocked that Claude told apart my print, my cursive, and my quick scratch, and still got so much right. Round one got every word right. Round two flagged five words across 14 pages. The notebook round produced two wrong words across four pages, and both are now rules.

‍

Try It Yourself

You need a Fulcra account (free to start with 5 GB) and Claude (or your agent of choice) with the Fulcra connector enabled.

1. Create a guide file in Fulcra. It can start empty.
2. Have Claude transcribe three or four of your handwritten pages, and ask it to flag any word it isn't sure about.
3. Review every line, and note any corrections.. Turn each correction into a rule in the guide, with the example that taught it.
4. Tell your agents to read the guide before they transcribe anything.
5. Run a harder batch and repeat. Keep going until the errors stop being about words.