Intro

For months now I’ve had a tab open in my browser on my main computer - Reclaiming the Web with a Personal Reader. I’ve read the article a couple of times and really like the idea of using a personal and extendable feed reader.

Finally around a week ago I cloned the repo from GitHub to check it out. After a couple of tweaks to the Makefile to make it work with my laptops configuration I was up and running. Someone else noticed the thing with the port being in use so they submitted a pull request on the repo so I didn’t have to.

Overall, I really liked it! I did discover a small typo which kept feeds from syncing when you clicked the sync button. Though to be fair I shouldn’t have needed to click sync, I think it should have imported the initial feeds when I ran make feed-load. But that’s just a little thing.

Python isn’t my strongest language so while I can follow along with the code OK I don’t feel like I can extend it as I see fit…

Desire to Extend

This led me to wanting to create my own version of the “personal feed reader”. Figuring this would be a good project for me to continue practicing my Go.

First, I made a list of packages that would get me to most of the basic functionality of the feed reader, leaving off Mastodon for now.

https://github.com/spf13/cobra
https://git.deanishe.net/deanishe/go-favicon
https://github.com/go-shiori/go-readability
https://github.com/go-co-op/gocron
https://github.com/mmcdole/gofeed
https://github.com/PuerkitoBio/goquery

I’m going to stick with using htmx and Bulma CSS as it’ll be a nice excuse to learn a bit about them.

To start my hacking away on the project I began with using the Cobra command line tools to stand up a basic app. From here I added skeleton commands, version, feed-load, serve.

version - doesn’t do much of anything at the moment.
feed-load - reads a local CSV file and prints out the contents.
serve - starts the HTTP server on 3000 and is slowly being extended to do other tasks.

What other tasks, you say?

While right now running serve will start up the web server, begin the scheduler and schedule a dummy job that just says hello. It will also load the feed CSV and do some parsing of the included feeds and then use readability on the feed links.

Databasing

My next step is to start working on the database. I need to do some research on properly doing migrations. The Go program/library migrate looks like it may be useful in that regard. At first we’ll likely be modeling tables after what’s already presented to us as using the base feedi as a design building block.

0_create_table.down.sql

DROP TABLE IF EXISTS feeds;

0_create_table.up.sql

CREATE TABLE feeds (
    id                  INTEGER   NOT NULL,
    user_id             INTEGER   NOT NULL,
    url                 VARCHAR,
    type                VARCHAR   NOT NULL,
    name                VARCHAR,
    icon_url            VARCHAR,
    created             TIMESTAMP NOT NULL,
    updated             TIMESTAMP NOT NULL,
    last_fetch          TIMESTAMP,
    raw_data            VARCHAR,
    folder              VARCHAR,
    etag                VARCHAR,
    modified_header     VARCHAR,
    filters             VARCHAR,
    mastodon_account_id INTEGER,
    PRIMARY KEY (
        id
    ),
    UNIQUE (
        user_id,
        name
    )
);

With the up/down files in ./migrations I should be able to add a command migrate and use the library to do the heavy lifting. I think, we’ll have to give it a whirl. Maybe I’ll write a small test project…

And my small test project was a success it seems. If you are interested here is the code.

package main

import (
	"database/sql"
	"fmt"
	"log"

	"github.com/golang-migrate/migrate/v4"
	"github.com/golang-migrate/migrate/v4/database/sqlite3"
	"github.com/golang-migrate/migrate/v4/source/file"
	_ "github.com/mattn/go-sqlite3"
)

func main() {
	// Connect to database
	db, err := sql.Open("sqlite3", "feeds.db")
	if err != nil {
		log.Panic(err)
	}
	// defer close
	defer db.Close()

	dbDriver, err := sqlite3.WithInstance(db, &sqlite3.Config{})
	if err != nil {
		fmt.Printf("instance error: %v \n", err)
	}

	fileSource, err := (&file.File{}).Open("file://migrations")
	if err != nil {
		fmt.Printf("opening file error: %v \n", err)
	}

	m, err := migrate.NewWithInstance("file", fileSource, "feeds.db", dbDriver)
	if err != nil {
		fmt.Printf("migrate error: %v \n", err)
	}

	if err = m.Up(); err != nil {
		fmt.Printf("migrate up error: %v \n", err)
	}

	fmt.Println("Migrate up done with success")

}