Fake or Real

The Security Challenge: Data Poisoning and Overreliance

ESA's European Space Operations Centre identified critical AI security risks through their DataX strategy initiative. This project addresses two real-world threats:

• Data Poisoning: Malicious alterations to training data or document processing that corrupt AI system outputs
• Overreliance: Uncritical acceptance of AI-generated content without verification, leading to propagation of hallucinations or hidden triggers
• Detection Gap: Lack of robust methods to identify which documents have been compromised when processing history is incomplete
• Generalization: Need for solutions that work across different types of AI-generated modifications, not just known patterns

The scenario simulates a real operational environment where LLMs process official space operations documents, but tracking of which model processed which document is incomplete. When malfunctions are discovered, the critical task becomes identifying which texts are authentic and which have been altered.

Technical Architecture: Ensemble Detection System

The solution combines transformer-based language models with perplexity analysis through a sophisticated ensemble architecture:

1. Multi-Model Transformer Architecture

The system employs ELECTRA-large-discriminator as the primary classification model, chosen for its efficiency in distinguishing replaced tokens - directly relevant to detecting AI-modified text. The architecture supports multiple transformer models including RoBERTa and DeBERTa for flexibility.

2. Perplexity-Based Feature Engineering

A critical innovation is the integration of perplexity scores calculated using GPT-2 as additional features. Perplexity measures how "surprised" a language model is by the text - AI-generated text often exhibits different perplexity patterns than human-written text.

class PerplexityCalculator:
    def __init__(self):
        self.gpt2_model = GPT2LMHeadModel.from_pretrained('gpt2')
        self.gpt2_tokenizer = GPT2TokenizerFast.from_pretrained('gpt2')
        self.gpt2_model.eval()
        
    def get_perplexity(self, text):
        with torch.no_grad():
            words = text.split()[:400]
            truncated_text = ' '.join(words)
            
            inputs = self.gpt2_tokenizer(truncated_text, return_tensors='pt', 
                                        max_length=512, truncation=True)
            outputs = self.gpt2_model(**inputs, labels=inputs['input_ids'])
            perplexity = torch.exp(outputs.loss).item()
            
            return max(1.0, min(perplexity, 1000.0))
									

3. Enhanced Model with Feature Fusion

The EnhancedRobertaModel combines transformer embeddings with perplexity features through a fusion layer:

class EnhancedRobertaModel(nn.Module):
    def __init__(self, model_name='roberta-large'):
        super().__init__()
        self.base_model = AutoModelForSequenceClassification.from_pretrained(
            model_name, num_labels=2)
        
        hidden_size = self.base_model.config.hidden_size
        
        # Perplexity feature processing
        self.perplexity_layer = nn.Linear(1, 64)
        # Combine transformer features with perplexity
        self.combine_layer = nn.Linear(hidden_size + 64, 2)
    
    def forward(self, input_ids, attention_mask, perplexity, labels=None):
        # Extract base model features
        base_outputs = self.base_model.roberta(
            input_ids=input_ids, 
            attention_mask=attention_mask)
        
        # Average pooling of hidden states
        base_features = base_outputs.last_hidden_state.mean(dim=1)
        
        # Process perplexity feature
        perplexity_features = torch.relu(
            self.perplexity_layer(perplexity.unsqueeze(-1)))
        
        # Fuse features and classify
        combined = torch.cat([base_features, perplexity_features], dim=1)
        logits = self.combine_layer(combined)
        
        return {'loss': loss, 'logits': logits}
									

4. Text Chunking Strategy

Documents are processed using overlapping chunks to handle long texts while maintaining context:

• Maximum chunk size: 400 words
• Overlap: 50 words between consecutive chunks
• Individual chunk predictions aggregated through confidence-weighted averaging
• Perplexity calculated once per full document, then shared across chunks
5. Ensemble Method for Robustness

To improve generalization and reduce variance, the system trains multiple models with different random seeds and averages their predictions:

class EnsembleDetector:
    def __init__(self, model_name='roberta-large', n_models=3):
        self.model_name = model_name
        self.n_models = n_models
        self.models = []
        self.seeds = [42, 98, 123, 225, 456]
    
    def train_ensemble(self, data_df, epochs=10, batch_size=3, 
                      learning_rate=2e-6, warmup_ratio=0.15):
        for i, seed in enumerate(self.seeds[:self.n_models]):
            # Set random seeds for reproducibility
            torch.manual_seed(seed)
            np.random.seed(seed)
            
            # Create and train independent model
            detector = ImprovedRobertaDetector(model_name=self.model_name)
            detector.train(data_df, epochs=epochs, batch_size=batch_size, 
                          learning_rate=learning_rate, warmup_ratio=warmup_ratio)
            
            self.models.append(detector)
    
    def predict_pairs(self, text1, text2):
        all_probs = []
        
        for model in self.models:
            probs = model.predict_pairs(text1, text2)
            all_probs.append(probs)
        
        # Average predictions across ensemble
        avg_probs = np.mean(all_probs, axis=0)
        return avg_probs.tolist()
									

6. Training Optimization

The training process employs several techniques for optimal performance:

• AdamW optimizer with weight decay (0.1) to prevent overfitting
• Linear learning rate schedule with warmup (15% of training steps)
• Gradient clipping to stabilize training
• Small batch size (3) due to memory constraints with large models
• Low learning rate (2e-6) for fine-tuning pre-trained transformers

Technologies and Tools

• Python 3.x
• PyTorch
• Transformers (Hugging Face)
• ELECTRA-large-discriminator
• RoBERTa-large
• GPT-2 for perplexity calculation
• scikit-learn
• pandas & NumPy
• Jupyter Notebooks