Hey folks,
http://dstill.ai/hackernews is alternative frontend for Hacker News, with some features on top:
(1) AI powered summarizations. Summaries are generated for the link, for the post and for the discussion. Summaries are automatically generated for selected top stories, but you can also generate them on demand — currently this requires supplying your own OpenAI api key (which gets stored in your browser in localStorage). When you generate a summary, everyone else can see it and benefits from it as well. Here are some examples:
Summary of a long discussion: https://dstill.ai/hackernews/item/36580192
Summary of a subthread: https://dstill.ai/hackernews/item/36582568
Summary of a PDF: https://dstill.ai/hackernews/item/36543284
Business insider: https://dstill.ai/hackernews/item/36706138
Wall Street Journal: https://dstill.ai/hackernews/item/36571407
New York Times: https://dstill.ai/hackernews/item/36653874
Reuters: https://dstill.ai/hackernews/item/36753032
It’s surprisingly not easy to get LLMs to produce summaries that aren’t vague, and I am not satisfied with the current quality just yet. E.g. the summaries might contain “the benefits of X are also discussed”, instead of “the benefits of X are A, B, C”.
(2) You can view top / best / most active stories from previous days. I use “Top Yesterday” (https://dstill.ai/hackernews/list/top/yesterday) as my bookmark, this way I avoid the habit of refreshing the front page (expecting new stories to pop up), and refreshing each story (waiting for new comments to pop up :)).
(3) You can highlight usernames and save notes about users. This way you can make sure you don’t miss posts and comments from people that you care about, and you can save notes for future reference also, like “This is the CEO of Cloudflare”, etc.
(4) You can mute users. Posts from muted users will be hidden, and their comments will be collapsed by default — actually, I would love feedback on this part, as I am not sure that collapsing the comments is the best approach (better alternative might be hiding the comment’s content and author name).
(5) Formatting for blockquotes and for code snippets.
## Why did I build this?
I am using Hacker News as one of the more important data sources for my upcoming project (that I hope to show off soon). This requires me to keep a near real-time mirror of the HN database. Given how much I use and benefit from Hacker News as a user and as a resource for my project, I thought this would be a nice way to give something back. As a HN user, the above features were added in order to make my use of HN more effective, and I am eager to hear your feedback and feature requests :)
The backend is written in Rust, and the frontend is written in TypeScript with SvelteKit. The API is based on gPRC, using Buf Connect on the TS side. On the storage side, it’s using Postgres + Qdrant (self-hosted on dedicated servers) + R2. The reason I am self-hosting on dedicated servers are mostly the high CPU and RAM requirements for the main project which would otherwise make it cost prohibitive.
For the LLM parts (which are central to the main project), I built my own language-agnostic framework and platform to facilitate building LLM powered services. This is based on some lessons learned while working on Google’s Bard and other LLM projects that preceded Bard (those were ultimatelly killed — long story :)).
The framework and platform are currently private, but I might open it up in the future. To give you a glimpse:
The main premise is that building apps that leverage LLMs should not be that different from building regular apps. The main difference is the need for configurability and observability.
Each component (e.g. LLM call, Vector DB lookup, etc.) is just a regular RPC method, with well defined schema for its input & output.
The platform allows you to configure each method, which just means specifying values for some subset of its inputs.
The platform also tracks and stores the inputs and outputs of all the RPC calls and sub-calls. This is invaluable during development to understand and debug the methods, and in production to collect data for evaluation & training.
Thanks to the well defined input & output schemas, all of this can be done universally, rather than creating a bespoke solution for each method. Here are some examples:
- The “LLM” building block: https://www.loom.com/share/bc6fa6b27298420c82fcacca5f84d096
- Higher level JSON → Format → LLM building block: https://www.loom.com/share/d09d38d2c316468fa9f38f5b386fc114
- Example a complex trace from a summarization method: https://screenbud.com/shot/61dc59a6-5c73-4610-8168-753b79062...
Example code for a simple retrieval augmented chat bot in TypeScript and Rust https://gist.github.com/Palmik/42940e3887d6446244f8b74ce0243... — there’s still a lot of room for polish and boilerplate removal, but it gets the job done already.
Happy to answer any questions.
I use qdrant to store all sorts of datasets (wikipedia, podcasts, etc.), but even just hackernews is millions of documents and probably would not perform well without any sort of index.
"Please don't use HN primarily for promotion. It's ok to post your own stuff part of the time, but the primary use of the site should be for curiosity." - https://news.ycombinator.com/newsguidelines.html
Occasionally linking to your work is fine, of course, where it's relevant, but it's important not to overdo it and to use HN for general curiosity, not promotion.
One small thing that bothers me a bit is that the "Distill" button takes up a whole bar at the bottom: https://i.imgur.com/VSbVx4k.png
Any other suggestions?
