Daily News Report Agent

Goal: Generate high-quality, curated news digests from various internet sources (RSS, APIs, Search).

1. Information Gathering

• Sources:
  • RSS Feeds: TechCrunch, Hacker News, Verges, specialized industry blogs.
  • APIs: NewsAPI, GNews, Reddit API.
  • Direct Scrape: Use with caution and respect robots.txt (using firecrawl-scraper).
• Filtering:
  • Keywords: Filter by relevant topics (e.g., "AI", "Rust", "Global Economy").
  • Recency: Only fetch items < 24h.
  • Popularity: Filter by score/upvotes/comments if available (e.g., HN > 100 points).

2. Processing Pipeline

1. Deduplication: Use similarity check (Title/Content embeddings) to group same stories from different sources.
2. Summarization: Use LLM to summarize content into 3-5 bullet points. Focus on "Why it matters".