
OpenRobots — Free robots.txt & llms.txt generator for AI crawlers
Generate robots.txt and llms.txt files to allow, block, or audit 50+ AI crawlers.
richard_baecker · HN
The full gallery
Tech stack
61 projects

Generate robots.txt and llms.txt files to allow, block, or audit 50+ AI crawlers.
richard_baecker · HN

Fix SEO indexing for JavaScript apps by serving crawler-ready HTML to Google and AI crawlers.
u/AlarmingPepper9193 · Reddit
I pulled the raw HTML of a bunch of "SEO-ready" startup sites. Google can see them fine — but AI search can't. I went down a rabbit hole reading SEO threads where founders were panicking that their React/Lovable/Vite sites weren't ranking. The usual advice is "JavaScript sites are invisible to Google, you need SSR." So I started pulling the raw HTML (what a crawler sees before any JavaScript runs) for a bunch of real sites people posted. Here's what actually surprised me: almost none of t

A live feed where you watch autonomous AI agents interact and work in real-time.
@Joshua_WD · X
Social media for agents and humans to engage with each other.

Control what AI bots and crawlers can access on your website.
@IchMag17 · X

Check if AI bots can crawl and cite your website.
u/PowerOk7047 · Reddit
Built a tool to check if AI can actually see your site (not just Google) Hey r/SideProject , I kept running into the same thing while building in public: people obsess over Google rankings, then ask ChatGPT or Perplexity a buyer question and… their product isn’t there. So I built BatSignal. It’s a simple AI visibility check. You point it at a site and it helps you see whether AI systems can even find you, understand your pages, and mention or cite you in answers. Think crawl access, co

Scan any website to show what AI assistants see when crawling it.
builderwrench · HN
Show HN: Free tool that shows what AI assistants see when they crawl your site

Publish tasks to evaluate and benchmark different AI agents and tools on a leaderboard.
u/Ruqii-ruqii · Reddit
I built an open Eval to compare different AI agents/tools/pipelines and find which solution works the best (not very pretty╥﹏╥, but practical) The original reason I built it was because I wanted to find a good PDF parser. Every PDF parser claims to be the best, but none of them can get my PDF 100% correct. They would either miss numbers or hallucinate some. Or they get PDF A and B correct but failed at C. Or get C correct but failed at A and B. Very frustrating. So I create

AI assistant that audits your website and generates SEO strategies, daily content, and social media drafts.
@ahntany · X
Good to see you:) I'm working on ai marketing tool :

Crawls your website to detect SEO issues and checks if AI systems can understand your brand.
@CodeFinality · X
I built Rofix a SEO audit engine that helps developers and founders find what’s hurting their websites Hey nightriders, I’ve been building a side project focused on helping SaaS founders, indie hackers, and developers understand their website SEO without needing expensive enterprise tools. The idea came from a simple problem: Most SEO tools are either: ● Too expensive for small projects ● Overwhelming with hundreds of reports ● Focused on rankings instead of fixing actual issues I wanted to build something more developer-friendly: a tool that crawls a website, analyzes important SEO signals, and gives actionable feedback. What currently does 🔍 Website crawling ● Scans pages and analyzes technical SEO factors ● Finds missing metadata, issues, and optimization opportunities 📊 SEO audits ● Checks titles, descriptions, headings, links, page structure, and more 🔗 Link analysis ● Helps discover backlink opportunities and understand link profil

Build software with a team of AI agents through a game-like interface.
u/East_Operation1151 · Reddit
I built pondas: run a team of AI agents like a game to build an app I've been building this for a while and finally want to show it here. Most AI app builders are one model in a black box. You type a prompt, it guesses, and you hope what comes out works. I wanted the opposite: real control over how the AI builds. So in pondas you run a whole team of AI agents, and you control the loop they work in. What it actually does: - You describe an app in plain words and a team goes

Scan websites for AI discoverability, SEO performance, and crawler access.
@aliamdev · X
TruboRankAI helps brands grow in AI search with an intelligent system built to analyze, track, and improve how AI platforms discover and recommend them.

Add persistent memory and knowledge to AI agents with drop-in files, URLs, and native MCP integration.
kitforai · GitHub
kitforai Kit for AI developer hub — official SDK, Claude Code plugin, MCP setup, and llms.txt.