Recent advancements in long-context language models (LLMs) have been highlighted, focusing on improving their ability to handle extended contexts. MemLong, a memory-augmented retrieval system, enhances LLMs by using an external retriever and a memory bank, significantly extending context length from 4,000 to 80,000 tokens. This development allows LLMs to outperform other state-of-the-art models on long-context benchmarks, even on a single 3090 GPU. Additionally, new methods such as Writing in the Margins, ReMamba, Dolphin, FocusLLM, and LongRAG have been introduced to improve the efficiency of processing long contexts in LLMs.