# Nginx Ingress GeoIP Analytics

This solution provides geographic visualization of traffic to your Kubernetes nginx ingress controller by extracting IP addresses from access logs and displaying them on a world map in Grafana.

## Architecture

```
┌─────────────────┐    ┌──────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Nginx Ingress  │───▶│   Promtail   │───▶│      Loki       │───▶│ GeoIP Enricher  │
│  (Access Logs)  │    │ (Log Scraper)│    │ (Log Storage)   │    │  (IP Analysis)  │
└─────────────────┘    └──────────────┘    └─────────────────┘    └─────────────────┘
                                                                             │
                                                                             ▼
┌─────────────────┐    ┌──────────────┐    ┌─────────────────┐    ┌─────────────────┐
│     Grafana     │◀───│  Prometheus  │◀───│  Pushgateway    │◀───│  Geo Metrics    │
│  (Visualization)│    │  (Metrics)   │    │ (Metrics Proxy) │    │    (Push)       │
└─────────────────┘    └──────────────┘    └─────────────────┘    └─────────────────┘
```

## Components

### 1. **Loki + Promtail** (Log Collection)
- **Loki**: Stores and indexes nginx access logs
- **Promtail**: Scrapes logs from nginx ingress pods and parses IP addresses

### 2. **GeoIP Enricher** (Custom Service)
- Queries Loki for recent access logs
- Extracts unique IP addresses
- Enriches IPs with geolocation data using ip-api.com
- Pushes geographic metrics to Prometheus via Pushgateway

### 3. **Prometheus + Pushgateway** (Metrics Storage)
- Stores geographic metrics with labels for country, city, latitude, longitude
- Provides time-series data for visualization

### 4. **Grafana Dashboard** (Visualization)
- World map showing request origins
- Pie chart of requests by country
- Time series of request rates
- Detailed geographic breakdown table

## Installation

### Deploy with Terraform

```bash
cd terraform

# Initialize Terraform
terraform init

# Plan the deployment
terraform plan

# Apply the configuration
terraform apply
```

This will deploy:
- Loki and Promtail for log collection
- Prometheus Pushgateway for metrics
- GeoIP enricher service
- Grafana with pre-configured dashboard
- All necessary Kubernetes resources

### Access the Services

After deployment, you can access:

- **Grafana**: https://grafana.junv.cc (admin/admin123)
- **Loki**: http://loki.logging.svc.cluster.local:3100
- **Pushgateway**: http://prometheus-pushgateway.prometheus.svc.cluster.local:9091

## Verification

```bash
# Check all components are running
kubectl -n logging get pods
kubectl -n prometheus get pods
kubectl -n grafana get pods

# Check GeoIP enricher logs
kubectl -n logging logs -l app=geoip-enricher -f

# Verify metrics are being pushed
kubectl -n prometheus port-forward svc/prometheus-pushgateway 9091:9091
# Visit http://localhost:9091/metrics and search for "nginx_geo"
```

## Configuration

### GeoIP Service Rate Limits

The solution uses the free ip-api.com service with these limits:
- 45 requests per minute
- 1000 requests per day

For production use, consider:
- Using MaxMind GeoLite2 database (local lookups)
- Implementing IP caching to reduce API calls
- Using paid geolocation services for higher limits

### Log Format

The nginx ingress is configured with a detailed log format:
```
$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_length $request_time [$proxy_upstream_name] [$proxy_alternative_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id
```

### Data Collection Frequency

- GeoIP enricher runs every 5 minutes
- Queries last 5 minutes of logs from Loki
- Processes up to 5000 log entries per cycle

## Troubleshooting

### No Data in Grafana

1. **Check if logs are being collected:**
   ```bash
   kubectl -n logging logs -l app.kubernetes.io/name=promtail
   ```

2. **Verify Loki has data:**
   ```bash
   kubectl -n logging port-forward svc/loki 3100:3100
   # Visit http://localhost:3100 and query: {job="nginx-ingress"}
   ```

3. **Check GeoIP enricher logs:**
   ```bash
   kubectl -n logging logs -l app=geoip-enricher -f
   ```

4. **Verify metrics in Pushgateway:**
   ```bash
   kubectl -n prometheus port-forward svc/prometheus-pushgateway 9091:9091
   # Visit http://localhost:9091/metrics and search for "nginx_geo"
   ```

### GeoIP Enricher Not Processing IPs

1. **Check if nginx logs are in expected format:**
   ```bash
   kubectl -n ingress-nginx logs -l app.kubernetes.io/component=controller
   ```

2. **Verify network connectivity to ip-api.com:**
   ```bash
   kubectl -n logging exec -it deployment/geoip-enricher -- curl -s "http://ip-api.com/json/8.8.8.8"
   ```

### High Memory Usage

1. **Adjust resource limits in geoip-enricher.yaml:**
   ```yaml
   resources:
     limits:
       memory: "512Mi"  # Increase if needed
   ```

2. **Reduce log query frequency:**
   - Edit the sleep time in enricher.py (default: 300 seconds)

## Security Considerations

1. **Network Policies**: Restrict GeoIP enricher network access
2. **Resource Limits**: Set appropriate CPU/memory limits
3. **RBAC**: Create minimal service account permissions
4. **Data Retention**: Configure Loki retention policies

## Monitoring

Monitor the solution with these queries:

```prometheus
# Enricher health
up{job="geoip-enricher"}

# Processing rate
increase(nginx_geo_requests_total[5m])

# Unique countries detected
count by (country) (nginx_geo_requests_by_country_total)
```

## Scaling

For high-traffic deployments:

1. **Horizontal scaling**: Increase GeoIP enricher replicas
2. **Caching**: Implement Redis cache for IP lookups
3. **Batching**: Process IPs in larger batches
4. **Local database**: Use MaxMind GeoLite2 for offline lookups

## Cost Optimization

1. **IP filtering**: Skip private/internal IP ranges
2. **Deduplication**: Cache recent IP lookups
3. **Sampling**: Process only a percentage of requests
4. **Regional focus**: Limit processing to specific regions

## License

This solution is provided as-is for educational and operational use. Please ensure compliance with your organization's security and privacy policies.