5632a3d641
RSS-based content discovery for GoToSocial instances. Features: - Multi-instance RSS feed processing - Docker deployment with .env configuration - Comprehensive statistics and monitoring - Production-ready with proper secret management
175 lines
4.8 KiB
Markdown
175 lines
4.8 KiB
Markdown
# GTS-HolMirDas 🚀
|
||
|
||
RSS-based content discovery for GoToSocial instances.
|
||
|
||
Automatically discovers and federates content from RSS feeds across the Fediverse, helping small GoToSocial instances populate their federated timeline without relying on traditional relays.
|
||
|
||
## Features
|
||
|
||
- 📡 **Multi-Instance RSS Discovery** - Fetches content from configurable RSS feeds across Fediverse instances
|
||
- ⚡ **Efficient Processing** - Configurable rate limiting and duplicate detection
|
||
- 🔧 **Production Ready** - Environment-based config, Docker deployment, health monitoring
|
||
- 📊 **Comprehensive Statistics** - Runtime metrics, content processing, and federation growth tracking
|
||
- 🐳 **Containerized** - Simple Docker Compose deployment
|
||
- 📁 **File-based Configuration** - Easy RSS feed management via text files
|
||
|
||
## How it Works
|
||
|
||
GTS-HolMirDas reads RSS feeds from various Fediverse instances and uses GoToSocial's search API to federate the discovered content. This approach:
|
||
|
||
- Maintains proper ActivityPub federation (posts remain interactive)
|
||
- Respects rate limits and instance policies
|
||
- Provides better content discovery for small instances
|
||
- Works alongside tools like FediFetcher for comprehensive federation
|
||
|
||
## Quick Start
|
||
|
||
```bash
|
||
# Clone the repository
|
||
git clone https://your-forgejo-instance.tld/user/gts-holmirdas
|
||
cd gts-holmirdas
|
||
|
||
# Copy configuration templates
|
||
cp .env.example .env
|
||
cp rss_feeds.example.txt rss_feeds.txt
|
||
|
||
# Edit configuration
|
||
nano .env # Add your GTS credentials
|
||
nano rss_feeds.txt # Customize RSS feeds
|
||
|
||
# Deploy
|
||
docker-compose up -d
|
||
|
||
# Monitor
|
||
docker-compose logs -f
|
||
```
|
||
|
||
## Configuration
|
||
|
||
### Environment Variables (.env)
|
||
|
||
```bash
|
||
# GTS Server Configuration
|
||
GTS_SERVER_URL=https://your-gts-instance.tld
|
||
GTS_ACCESS_TOKEN=your_gts_access_token
|
||
|
||
# Processing Configuration
|
||
MAX_POSTS_PER_RUN=25 # Posts per feed per run
|
||
DELAY_BETWEEN_REQUESTS=1 # Seconds between API calls
|
||
LOG_LEVEL=INFO # Logging verbosity
|
||
|
||
# RSS Configuration
|
||
RSS_URLS_FILE=/app/rss_feeds.txt # Path to RSS feeds file
|
||
|
||
# Optional: Monitoring
|
||
HEALTHCHECK_URL=https://hc-ping.com/your-uuid
|
||
```
|
||
|
||
### RSS Feeds (rss_feeds.txt)
|
||
|
||
```
|
||
# Example RSS feeds - customize for your interests
|
||
# homelab
|
||
https://mastodon.social/tags/homelab.rss
|
||
https://fosstodon.org/tags/homelab.rss
|
||
|
||
# selfhosting
|
||
https://mastodon.social/tags/selfhosting.rss
|
||
https://infosec.exchange/tags/selfhosting.rss
|
||
|
||
# Add your preferred instances and hashtags
|
||
```
|
||
|
||
## Access Token Setup
|
||
|
||
1. Login to your GoToSocial instance
|
||
2. Go to Settings → Applications
|
||
3. Create new application with scopes: `read`, `read:search`, `read:statuses`
|
||
4. Copy the access token to your `.env` file
|
||
|
||
## Statistics Output
|
||
|
||
```
|
||
📊 GTS-HolMirDas Run Statistics:
|
||
⏱️ Runtime: 0:04:14
|
||
📄 Total posts processed: 45
|
||
🌐 Current known instances: 2519
|
||
➕ New instances discovered: +3
|
||
📡 RSS feeds processed: 25
|
||
⚡ Posts per minute: 10.6
|
||
```
|
||
|
||
## Resource Requirements
|
||
|
||
- **Memory**: ~200-500MB depending on feed count
|
||
- **CPU**: Minimal (mostly I/O bound)
|
||
- **Storage**: <100MB for application, plus log storage
|
||
- **Network**: Depends on RSS feed count and frequency
|
||
|
||
## Deployment Options
|
||
|
||
### Docker Compose (Recommended)
|
||
```bash
|
||
docker-compose up -d
|
||
```
|
||
|
||
### Standalone Docker
|
||
```bash
|
||
docker build -t gts-holmirdas .
|
||
docker run -d --env-file .env \
|
||
-v ./data:/app/data \
|
||
-v ./gts_holmirdas.py:/app/gts_holmirdas.py:ro \
|
||
-v ./rss_feeds.txt:/app/rss_feeds.txt:ro \
|
||
gts-holmirdas
|
||
```
|
||
|
||
## Monitoring
|
||
|
||
- **Logs**: `docker-compose logs -f`
|
||
- **Health**: Optional Healthchecks.io integration
|
||
- **Statistics**: Built-in runtime and performance metrics
|
||
- **Resource Usage**: Docker stats or container monitoring tools
|
||
|
||
## Troubleshooting
|
||
|
||
### Common Issues
|
||
|
||
**No posts processed**: Check access token permissions and RSS feed URLs
|
||
|
||
**Rate limiting errors**: Increase `DELAY_BETWEEN_REQUESTS` or reduce feed count
|
||
|
||
**High memory usage**: Reduce `MAX_POSTS_PER_RUN` or feed frequency
|
||
|
||
**Container won't start**: Verify `.env` file format and required variables
|
||
|
||
### Debug Mode
|
||
|
||
```bash
|
||
# Enable debug logging
|
||
echo "LOG_LEVEL=DEBUG" >> .env
|
||
docker-compose restart gts-holmirdas
|
||
```
|
||
|
||
## Contributing
|
||
|
||
1. Fork the repository
|
||
2. Create a feature branch
|
||
3. Make your changes
|
||
4. Test thoroughly
|
||
5. Submit a pull request
|
||
|
||
## Related Projects
|
||
|
||
- [FediFetcher](https://github.com/nanos/fedifetcher) - Fetches missing replies and posts
|
||
- [GoToSocial](https://github.com/superseriousbusiness/gotosocial) - Lightweight ActivityPub server
|
||
- [slurp](https://github.com/VyrCossont/slurp) - Import posts from other instances
|
||
|
||
## License
|
||
|
||
MIT License - see LICENSE file for details.
|
||
|
||
## Acknowledgments
|
||
|
||
- Inspired by the [HolMirDas](https://github.com/aliceif/HolMirDas) concept
|
||
- Built for the GoToSocial community
|
||
- RSS-to-ActivityPub approach inspired by Fediverse discovery challenges
|