Mintlify has introduced a new feature allowing companies to create 'llms.txt' files, which are designed to facilitate the interaction between websites and large language models (LLMs). This initiative aims to help businesses maintain two versions of their documentation: one for human users and another optimized for LLMs. The llms.txt Generator, developed in collaboration with Firecrawl, enables users to concatenate any website into a single text file that can be processed by LLMs. The tool crawls the entire website and extracts data using the gpt-4o-mini model, making it easier for LLMs to read and understand the content. Additionally, users can generate their own llms.txt files to enhance the AI's contextual understanding of their websites.
A tool for creating LLM.txt files, or instructions, for AI context and prompting. You can extract any website site into a nicely prepared text file ready to ‘@‘ Serious secret weapon stuff by the Genius’s at @firecrawl_dev 🔥 Example: https://t.co/z7uitnYqEA https://t.co/NRjQ05rTGi
Lessons from creating an llms.txt file An llms.txt file is a way to tell a LLM about your website. In the .txt file, you include links to other files with info to learn more. - the llms.txt file isn't the file you send to an LLM, you use it to generate a llms .md file - it's…
Generate llms.txt files with Firecrawl 🔥 A llms.txt file can make your website easier to read for LLMs. Check it out: https://t.co/vaTgdqAcXX