Show older

before i shut down my free bot hosting service, i was hosting 108 bots. that means my server had 108 open connections listening for replies at all times. i don't know how many it can handle, but the answer's at least 108 :blobcatfingerguns:

i have no idea how many people are going to use this, though, so it might not go so well. each reply listener needs to be its own process, too... i might have to either get a separate server just for this or upgrade my current server to the prohibitively expensive next tier

in other words, when this goes live, i'm kinda hoping for people clicking on those donation links so i can upgrade the server! :blobbun_mlem:

this is by the people (henlo) for the people (you!) so if there's anything at all you'd like to see, let me know!

what do you want in an ebooks bot? what annoys or disappoints you about the current ones? if you could program, what custom functionality what you give your own one? what do you like or hate about webapps (like mastodon and gmail) that could apply here? what is your favourite breed of rabbit?

allowing users to get their bot to post once every five minutes is probably a bad idea, right

i'm excited to work on the backend but the frontend still isn't done, there's just so much to do!! :bunrainbow:

the more hardcore lynnesbian fans may note that fedibooks looks rather similar to curious greg (no not :greg:​, i mean cg.lynnesbian.space)

i did take inspiration from that design, which in turn took inspiration from mastodon's web UI. however, i made a few changes, most noticeably to the buttons, to give it its own unique identity. i also decided to go a little bit brutalist (and i really do mean a little bit) by leaving everything with sharp, square edges, flat colours, monotone icons, and so on.

who says programmers can't design frontends? :chloeblob:

you may also notice that fedibooks used to be a different project. i've abandoned the old fedibooks and moved to this one instead. in retrospect, the old fedibooks concept was way too ambitious. it was around the time i was considering how to create a simple UI for displaying complex decision trees that i realised i was in too deep.

why create fedibooks? (long) 

I've been working on ebooks bots for a long time. My first project, mstdn-ebooks, dates back to October 2018. It's a much less ambitious project than FediBooks, but maintaining and developing it has still been a formidable undertaking. It's changed a lot since the initial version, and has worked very well for a long time.

Using mstdn-ebooks is nowhere near as easy as using FediBooks. There's a long guide you'll need to follow, and you'll have to install some programs like Python and possibly Git to get working. It runs on your own computer, meaning that if you put it into sleep mode, it stops working. Updating it to the latest version is also your responsibility, and the error messages it gives aren't exactly user friendly.

To help with this issue, I decided to create a Patreon where I offered a $2.50 per month tier to host up to three bots for you. I would take care of configuration, maintenance, updating, et cetera. This worked pretty well up until recently.

A new change in Mastodon and Pleroma adds something called authenticated fetches. Reffered to as "authorised mode" by Mastodon, this new feature prevents unauthorised instances and programs from accessing the API. In other words, if instance A blocks instance B, then instance B won't be able to see instance A's public posts anymore. However, this also means mstdn-ebooks can't see your posts. This can be fixed, however, by requesting permission from instance B. So what's the problem?

I would need to ask every single person I run a bot for to click a link I send to them, and then send me back the authentication code they received. This is messy and annoying, and greatly tedious. Furthermore, mstdn-ebooks' code base is starting to show its age (I used to be even worse at programming, believe it or not), and I thought this gave me a good motivation to create a new project.

When FediBooks needs permission from instance B, all you need to do is click "authorise". No copy and pasting codes and sending them in a Patreon DM. This is much cleaner and easier for the end user, which is what this is all about!

FediBooks is one of the biggest projects I've ever taken on, and it's been wonderful working on it. I hope you enjoy it!

hmm what if someone tries to make a bot that learns from someone with in their profile

when you ask me a question like "why pour all this time and energy into creating a free service" the answer always boils down to "i want to make people happy"

that's all there is to it really. my current dream job (not that i'm really capable of working) is a computer science lecturer, because i've seen how dry and boring compsci lectures are and i think i can do better. i want people to learn and have a good time doing it!

if i'm making other people happy, i'm almost always happy! i just need to remember to not let that get in the way of my own health.

okay so i think this is what the database is gonna look like

yes this is an ERD what about it

i just realised that i actually used a skill i learned at university today

holy shit

university taught me something i used in a real life scenario

if you had have told past me that i would get so dedicated to a passion project that i made an ERD for it, i would say "really? fuck"

hmm i could either work on more frontend stuff or try to convert this ERD into SQL

i might work on the SQL stuff since it's been a while :bunrainbow:

known issue: fedibooks will stop working after 4,294,967,295 people connect their accounts

looking over this SQL is writing is making me feel very Powerful

i could change fedibooks to only allow one user per instance and all it would take is a single word

i could cause privacy issues and wasted disk space by just deleting one rule

i could change a single column's data type and break pleroma compatibility

i actually kinda like working with databases, i just don't like database theory

i fixed up some of the issues with the entity relationship diagram and it's looking pretty solid now :chloeblob:

fedibooks can store (2^64)-1 posts

that is a lot of posts!!

i'd probably run out of disk space before getting anywhere near running out of post IDs :blobbun:

a difficult thing to determine is how high should i set the post length limit

my instance has it set to 65535 to ensure i never run into issues, masto defaults to 500, pleroma defaults to 5000

i guess this is supposed to be a microblogging platform so it doesn't make too much sense to allow them to be overly large, and that does technically expose me to some jerk filling my DB with posts that are gigabytes long

65535 it is :blobbuncoffee2:

the frontend stuff is getting along well

as for the backend stuff, well, there's more SQL than python, and i haven't even hit 100 lines yet, so

okay, the frontend stuff is getting close to being ready. when fedibooks first goes live, some features won't be ready yet. the basic stuff will all work just fine, though!

the backend stuff is gonna be a lot harder than the frontend, but i'll hopefully have it done before too long!

also worth noting: the whole site works without javascript :blobcatfingerguns:

there are a few features that will require it, but so far it's 100% JS free

which is nice because i'm bad at JS

i like javascript and i think it's important, but i also like avoiding it where possible to make everything as lightweight as possible

what do you typically call the "@user@instan.ce" thing? i call it a handle

one of the best parts of webdev is having to implement input validation on both the client and the server

hmm i wonder how i implemented input sanitisation in my last project

...oh! i didn't!

well time to shut cg.lynnesbian.space down forever

wait a minute never mind i overreacted it DOES sanitise inputs

phewwwwwwww

okay curious greg is back up so you can use it again now

even though it's really buggy and shit

the bot creation process is done now. it doesn't handle errors gracefully yet, but apart from that, it's ready!

this means you can now create a bot with fedibooks 0u0

now for the learning from posts part :blobbun:

things are coming along very well! you can create a bot and add accounts for it to learn from. there's a few basic things left, and then it's on to the actual ebooks stuff - downloading and making posts!!

531 (significant) lines of HTML, 328 lines of python, 174 lines of CSS, 76 lines of SQL, 1 tired bune

it's been a while i last updated this thread, because it's also been a while since i did any major work on fedibooks. i've kinda pushed myself way too hard on it and i'm taking it more slowly now.

that said, i do have some new features to show off! firstly, fedibooks is now capable of scraping user posts. this is one of the key steps towards implementing full ebooks functionality. the first screenshot shows that fedibooks has successfully scraped all 63 of this account's posts.

secondly, there's now a nifty little bug report form for those of us without github accounts, shown in the second screenshot.

fedibooks is now capable of:
. authenticating with mastodon/pleroma
. scraping user posts
. generating and submitting its own posts

this means it can do nearly everything mstdn-ebooks can, and all that's left is replying!! :toot:

uh oh, looks like you're out of gems! your ebooks bot will stop posting until you have at least 3 more.

[ wait for gems to recharge (3h27m) ]
[ watch 3 ads ]
[ buy pack of 15 gems ($4.99) ]

there are a few minor features left now, and one major feature - replying. once that's done, fedibooks will be complete.

unfortunately, testing the reply functionality requires me to actually deploy fedibooks to a website, since i can't test it with localhost. this means it's going to be a bit of a hassle to implement.

however, i've looked at what needs to be done for reply functionality, and it seems pretty easy with web push - mastodon will automatically send a request to fedibooks with some info, which fedibooks can then reply to. it should be pretty simple 0u0

i want to implement reply functionality and bot settings, and then i'll release the first version of fedibooks for all to use! :toot:

once fedibooks is released, i'll be shutting down my paid ebooks hosting service in a week (since it's available for free now). i'll implement an "import from mstdn-ebooks" feature to ease the transition.

thank you so much to everyone who supported me on patreon!

i'm really excited for this. this is the first time i've ever actually managed to finish a major project! i often start things that are way too ambitious and never manage to finish them, but this time i did it!!

this is the screen that appears when you try to do something i haven't made yet

i'm about to start testing the reply functionality for fedibooks 0u0

fedibooks is now live, by the way! i've disabled signups for now, but in the mean time, you can explore the site: fedibooks.com/

well i've run into a bit of an issue...

the bot creation process works on my small laptop and pet's laptop, but not on my big laptop. i have no idea why, it's the same code and everything, and my small laptop has the same firefox addons as the big one. i'm reading over the code and i really don't see how this bug works... and why does it only happen on one laptop??

okay i found the issue

this specific install of firefox wants the favicon.ico, and when it can't find it, it requests fedibooks' home page

because your browser is requesting the home page, fedibooks assumes you've finished creating the bot, and resets your progress counter thingy to 1.

this breaks multi-step forms.

okay!! fedibooks is now live and in what you might call early access.

fedibooks.com

i don't know how well it's going to hold up, because i haven't been able to do much serious testing on it, but i guess we'll all find out together!!

Show newer

@lynnesbian Clearly you need more one-character typo fixes so boost those numbers.

@lynnesbian
if you create two bots that pull from the same sources, can you then make them learn from each other?

@lynnesbian

can't wait!

what's self hosting going to be like compared to e-books? easier? harder? about the same?

@mal it's gonna be a lot easier! you won't need to download or install everything, the whole thing is done through the website. everything from replies to post frequency can be set without editing config files or anything like that. it'll be hosted on my server, so you don't need to keep your laptop running 24/7

@lynnesbian

oh ok, I misunderstood, I though since you were discontinuing the patreon thing that fedibooks was a self hosted thing, gotcha, this is cool though! I'm really glad it sounds to be nearly ready for prime time!

@lynnesbian if you haven’t found out yet, the Right Way to solve this is to make your forms POST and check for POST requests - this is essentially why different methods exist (GET requests should in theory never change server side state)

Show newer

@lynnesbian http get shouldn’t have side effects. find some ofher way to advance the counter

@lynnesbian that is, if you’ve got a multistep process, it should only advance with a form post. that’s application state change, and having that anywhere near GET always causes problems similar to yours because pretty much all software assumes GET is safe to do and won’t change application state.

Show newer
Show newer

@lynnesbian huh! you're using the webpush code, that's neat! If you notice any problems with it or way the docs can be improved, please let me know - don't think it's seen much use so far so it's probably Not Great

@lynnesbian (esp. if you notice that like push subs stop working for some reason because I've been like, wondering what happens if the url stops working for a while or whatever, does it clear the subscription, &c)

@lynnesbian Looks nice but remove Google Fonts and FontAwesome.com,please.People here care about their privacy and these fonts are all free to host on your own server.

Show newer

@lynnesbian watching your process on this has been inspiring. you literally turned a shitty terrifying situation into something great.
i hope my commentary along the way has been more helpful than annoying

@lynnesbian one zoe telling bune she's so proud of her and what a good job

Sign in to participate in the conversation
Lynnestodon

@lynnesbian@fedi.lynnesbian.space's anti-chud pro-skub instance for funtimes