Building the UK Food Safety Explorer: Government Data Made Accessible

Ever wondered which foods in the UK are safest from pesticide residues? Government data holds the answers, but it's buried in complex spreadsheets that most consumers can't navigate. That's why I built the UK Food Safety Explorer – transforming official DEFRA monitoring data into an accessible, consumer-friendly platform.

The Problem: Hidden in Plain Sight

The UK government regularly monitors food for pesticide residues through the Department for Environment, Food and Rural Affairs (DEFRA). Their Expert Committee on Pesticide Residues in Food (PRiF) publishes quarterly reports with detailed test results. The Q4 2024 report contained a treasure trove of data:

• 856 food samples tested across Great Britain and Northern Ireland
• 27 different food categories from beef to grapes
• 61 unique pesticides monitored
• 94 separate data sheets in complex ODS spreadsheet format

The problem? This vital public health information was essentially inaccessible to the people who need it most: UK families making daily food choices.

Project Vision: Clean 5 & Watch List

I envisioned a simple, intuitive tool that could answer questions like:

• "Which foods are safest for my family?"
• "Should I buy organic versions of certain foods?"
• "How do contamination rates compare across different categories?"

The solution needed to be:

• Consumer-focused: Clear safety ratings, not technical jargon
• Actionable: Specific recommendations, like a "Clean 5" list
• Transparent: Full government data attribution and methodology
• Mobile-friendly: Accessible while shopping

Technical Architecture

Data Processing Pipeline

The biggest challenge was transforming 94 complex CSV files into a unified, queryable dataset:

def process_all_data():
    """Main processing function"""
    all_samples = []
    food_summaries = {}
    
    # Process all BNA files (sample details)
    bna_files = list(RAW_DATA_DIR.glob("*_BNA.csv"))
    
    for filepath in bna_files:
        samples = process_bna_file(filepath)
        all_samples.extend(samples)
        
        # Group by food and region
        if samples:
            food_name = samples[0]['food_name']
            region = samples[0]['region']
            key = f"{food_name}_{region}"
            
            if key not in food_summaries:
                food_summaries[key] = []
            food_summaries[key].extend(samples)

The processing pipeline handles:

• Data cleaning: Standardizing column names and formats
• Pesticide parsing: Extracting chemical names, levels, and MRL values
• Regional aggregation: Combining GB and NI data appropriately
• Quality validation: Ensuring data integrity throughout

Safety Scoring Algorithm

I developed a comprehensive scoring system that considers multiple factors:

def calculate_food_safety_score(samples: List[Dict[str, Any]]) -> Dict[str, Any]:
    """Calculate comprehensive safety score for a food item"""
    contamination_rate = (samples_with_residues / total_samples) * 100
    violation_rate = (samples_with_violations / total_samples) * 100
    
    # Safety score calculation (0-100, higher is safer)
    safety_score = 100
    
    # Reduce score based on contamination rate
    safety_score -= contamination_rate * 0.5
    
    # Heavily penalize MRL violations
    safety_score -= violation_rate * 5
    
    # Bonus for organic samples
    organic_bonus = min(10, (organic_samples / total_samples) * 10)
    safety_score += organic_bonus
    
    return {
        'safety_score': round(safety_score, 1),
        'safety_rating': calculate_rating(safety_score),
        'contamination_rate': round(contamination_rate, 1),
        'violation_rate': round(violation_rate, 1)
    }

The algorithm weighs:

• Contamination frequency: How often residues are detected
• MRL violations: Samples exceeding Maximum Residue Limits
• Organic availability: Bonus points for foods with organic options
• Sample size: Confidence adjustments based on testing volume

Frontend Implementation

Modern Next.js Architecture

The application uses Next.js 15 with TypeScript for type safety and performance:

// API endpoint for food rankings
export async function GET() {
  try {
    const dataPath = path.join(process.cwd(), 'public/api/food-rankings.json');
    const data = JSON.parse(fs.readFileSync(dataPath, 'utf8'));
    
    return NextResponse.json(data);
  } catch (error) {
    return NextResponse.json({ error: 'Failed to load food data' }, { status: 500 });
  }
}

Interactive Dashboard Components

The homepage features a comprehensive dashboard with:

// Key statistics cards
<div className="grid md:grid-cols-2 lg:grid-cols-4 gap-6">
  <StatCard 
    icon={<Shield />}
    title="Foods Tested"
    value={stats.total_foods_tested}
    subtitle={`${stats.total_samples.toLocaleString()} total samples`}
  />
  <StatCard
    icon={<CheckCircle />}
    title="Clean Rate" 
    value={`${(100 - stats.contamination_rate).toFixed(1)}%`}
    subtitle="samples with no residues"
    color="green"
  />
  {/* Additional cards... */}
</div>

Consumer-Friendly Features

Clean 5 Rankings: The safest foods based on safety scores

{stats.cleanest_foods.slice(0, 5).map((food, index) => (
  <div key={food.name} className="bg-green-50 rounded-lg border border-green-200">
    <span className="font-bold text-green-700">#{index + 1}</span>
    <h3 className="font-semibold">{food.name}</h3>
    <p className="text-green-600">{food.safety_rating}</p>
    <p className="text-2xl font-bold text-green-700">{food.safety_score}/100</p>
  </div>
))}

Watch List: Foods with higher contamination rates where organic alternatives might be preferable.

Key Discoveries from the Data

The Clean 5 - Safest UK Foods

1. Beef (100/100) - Zero residues detected in all 25 samples
2. Milk (100/100) - Clean across all 71 samples tested
3. Beetroot (100/100) - Minimal contamination (3.2% rate)
4. Honey (100/100) - Only 1 pesticide found in 48 samples
5. White Fish (100/100) - Completely clean across 43 samples

Watch List - Consider Organic

1. Aubergine (25.9/100) - 74.1% contamination rate, 6 different pesticides
2. Grapes (37.9/100) - 89.7% of samples had residues
3. Limes (32.5/100) - 85% contamination rate with MRL violations
4. Sweet Peppers (47.3/100) - 13 different pesticides detected
5. Eggs (32.1/100) - Surprisingly high violation rate (12.5%)

Surprising Insights

• Only 1.3% of samples exceeded legal limits (MRL violations)
• 56.9% of UK food has zero detectable pesticide residues
• Organic products consistently scored higher where available
• Regional differences exist between GB and Northern Ireland
• Animal products (beef, milk, fish) are consistently cleanest

Development Challenges and Solutions

Challenge 1: Complex Data Structure

Problem: 94 separate CSV files with inconsistent formatting Solution: Built robust parsing that handles missing headers and variable column structures

Challenge 2: Pesticide Name Extraction

Problem: Residue data was in text format like "pesticide 0.02 (MRL = 0.9)" Solution: Created regex patterns to extract chemical names, detected levels, and MRL values

pattern = r'([a-zA-Z][a-zA-Z0-9\-\s\(\)]+?)\s+([\d\.]+)\s*\(MRL\s*=\s*([\d\.]+\*?)\)'
matches = re.findall(pattern, residue_text)
for match in matches:
    pesticide = match[0].strip()
    detected = float(match[1])
    mrl = float(match[2].rstrip('*'))

Challenge 3: Performance with Large Dataset

Problem: Processing 856 samples with complex relationships Solution: Implemented efficient caching and pre-computed JSON APIs

Challenge 4: Consumer Communication

Problem: Making technical data accessible without losing accuracy
Solution: Created clear safety ratings while maintaining full transparency about methodology

Technical Implementation Highlights

API Design

// RESTful endpoints for different data views
/api/stats          // Summary statistics
/api/foods          // All food rankings  
/api/foods/[slug]   // Individual food details
/api/pesticides     // Pesticide database

Responsive Design

The interface adapts seamlessly from desktop to mobile, essential for on-the-go food safety checks:

<div className="grid md:grid-cols-2 lg:grid-cols-4 gap-6">
  {/* Cards stack vertically on mobile, spread horizontally on desktop */}
</div>

Search Functionality

Real-time food lookup with intelligent slug generation:

const handleSearch = () => {
  if (searchTerm.trim()) {
    const slug = searchTerm
      .toLowerCase()
      .replace(/\s+/g, '-')
      .replace(/[^a-z0-9-]/g, '');
    window.location.href = `/foods/${slug}`;
  }
};

Public Health Impact

Making Government Data Actionable

This project transforms abstract government monitoring into concrete consumer guidance:

• Families can make informed shopping choices
• Health professionals have accessible data for dietary advice
• Policy researchers can identify contamination trends
• Journalists can access verified government data easily

Transparency and Trust

By maintaining full attribution to DEFRA and explaining our methodology, the platform builds trust while making complex data digestible.

Future Enhancements

Planned Features

• Historical trends: Multi-quarter comparison when more data becomes available
• Regional maps: Geographic visualization of contamination patterns
• Shopping lists: Personalized recommendations based on dietary preferences
• Alerts: Notifications when new government data is published

Technical Roadmap

• Progressive Web App: Offline functionality for mobile users
• Advanced filtering: Search by specific pesticides or contamination levels
• Data export: CSV downloads for researchers and journalists
• Accessibility: Enhanced screen reader support and keyboard navigation

Lessons Learned

1. Government Data is Goldmine

Official datasets contain incredible insights, but accessing and processing them requires significant technical expertise. Creating bridges between government data and public understanding is valuable work.

2. User Experience Matters Most

No matter how sophisticated the analysis, if users can't easily understand and act on the results, the project fails. The "Clean 5" concept resonated much better than technical contamination percentages.

3. Transparency Builds Trust

By clearly explaining our methodology, citing all sources, and acknowledging limitations, the platform builds credibility essential for public health tools.

4. Mobile-First is Essential

Food safety decisions happen in supermarkets, not just at home. The mobile experience needed to be as polished as the desktop version.

5. Performance at Scale

Processing 856+ samples with complex relationships taught valuable lessons about data structure optimization and caching strategies.

Technical Stack Summary

Frontend: Next.js 15, TypeScript, Tailwind CSS, Lucide Icons Backend: Next.js API Routes, Node.js file system operations
Data Processing: Python, pandas, custom parsing algorithms Deployment: Vercel, GitHub private repository APIs: RESTful JSON endpoints with proper error handling

Conclusion

The UK Food Safety Explorer demonstrates how technical skills can create genuine public value. By transforming complex government data into an accessible consumer tool, the project bridges the gap between official monitoring and family food choices.

The most rewarding aspect was seeing how data visualization could make important health information actionable. When someone can quickly check whether they should prioritize organic produce for their children, or feel confident about conventional options, that's technology serving its best purpose.

As government datasets continue growing in size and complexity, developers who can translate bureaucratic data into human understanding will play increasingly important roles in public health communication.

The platform is now live and helping UK families make informed food safety decisions every day.

---

Data source: DEFRA Expert Committee on Pesticide Residues in Food (PRiF) Q4 2024 Report. This platform is for informational purposes only and does not replace professional dietary advice. Always consult healthcare professionals for specific nutritional guidance.

Technical Note: The safety scoring algorithm and data processing methodology are fully documented in the project repository. The platform processes official government data without modification, applying transparent statistical analysis to generate consumer-friendly rankings.