have you ever noticed that someone post a screenshot of text without providing an image description, preventing you from being able to read it? or maybe they posted a huge amount of text that just can't possibly be described with the image description feature?

simply tag @OCRbot in a direct reply to the image. it will download the image and scan it using tesseract OCR to output the text contained in the image!

because OCRbot runs on fedi.lynnesbian.space, it has a character limit of 65535, so even the longest images should work OK!

check the reply to this image to see it parsing the attached screenshot!

some caveats:
- it's not perfect, and it will often make small mistakes
- some images won't work at all. tesseract works best with black text on a white background, or other simple layouts.
- OCRbot won't work with handwriting.

most importantly, OCRbot is *not* a substitute for captioning your images. it's imperfect, it trips up on anything but the simplest text screenshots (with basic fonts and single colour background images), and it puts the OCR output *in the replies*, meaning that people have to do an extra step to see the text output of an image.

more things it can't do:

- it doesn't know how to handle images (such as profile pictures within posts) at all. it'll usually know to ignore them but sometimes it'll try to read them as text.
- it can't handle complex text. even things as simple as italics can trip it up. text on angles, handwriting, etc. won't work.
- it doesn't do videos, not even the thumbnails. this includes GIFs.
- if your image doesn't make it to my instance, it won't work. for example, if you have a really slow server, and it takes more than 30 seconds for an image to get from your server to mine, it won't work.
- it can't read followers-only posts.

so please, please caption your images. OCRbot is a last resort for the vision impaired and others who want to get the text out of your image where you didn't provide it. OCRbot doesn't work anywhere near as well as an able-bodied human can. if the text is upside down, OCRbot doesn't have a chance, but you and i do.

Follow

and for the moment, it only does english. it works best with english, it'll probably trip up on accented letters, and it has no chance at arabic or japanese or anything else that doesn't use the latin alphabet.

if you have any suggestions, bug reports, requests, or anything else, you can either send me a direct message or open a github issue: github.com/Lynnesbian/OCRbot/i

ladies and/nand/or/xor/nor gentlemen...... i have solved the problem of accessibility forever.

in all seriousness though, this took me less than two days to make, and most of that time was spent watching youtube and playing factorio. the entire thing was put together from modified mstdn-ebooks code in a few hours. it's nothing special

OCR Output 

@oreolek i woulnd't really say that, it only works with the most basic text

OCR Output 

-Looks at the above toots- Time to build an app that posts captcha images to the Fediverse, tags OCRBot, then parses the reply to fill out the captcha.

@lynnesbian The combination of extremely useful tool and huge potential for hilarity makes this the most Lynne thing ever. I salute you.

@lunaterra i've noticed a lot of people posting screenshots of text without captions so i already kinda wanted to fix that somehow

then someone posted a huuuge tumblr screenshot and someone else asked for a caption

i thought "i can do that with OCR" and used OCR to generate the text, cleaned it up by hand, then replied with it

then i thought... why not make a bot for it? :blobcatderpy:

OCR Output 

@lynnesbian I had this exact same experience using Tesseract for videogame text. It surprised me a little bit!

@lynnesbian Lynne, don't sell yourself short, this is an awesome bot that I wouldn't even know how to begin making myself, and you're awesome for being able to make it :P

@lynnesbian
It is everything special for those who need it. Thank you!

@lynnesbian This is awesome!

And btw, never judge something's worth by its size or how much time you put into it or whatever.

It might've only taken two days to hack together the code, but the service it provides makes it much, much more valuable than a lot of things that took days, weeks, months, or even years to make.

@lynnesbian lynne, it made ZERO mistakes transliterating the definition of besto in lojban and I'm pretty sure *I* wouldn't be able to type that without mistakes.

@lynnesbian @OCRbot I need to try this, sorry.

(Text says something like “now I need to try with some italian, just because” the “così” is quite colloquial, chosen because of the accent.)

OCR Output 

@lynnesbian

and then realized that it was probably a bad choice, as ì is probably very close to i; I should have tried with some è or à. maybe later…

Sign in to participate in the conversation
Lynnestodon

@lynnesbian@fedi.lynnesbian.space's anti-chud pro-skub instance for funtimes