Skip to content

Duplication Pattern Analysis

Overview

Analysis of the Playcast codebase reveals significant patterns of code duplication across applications, particularly in backend services, build configurations, and common functionality. This duplication increases maintenance overhead, creates inconsistencies, and makes it difficult to implement platform-wide changes.

Major Duplication Patterns

1. Backend Service Initialization Pattern

Pattern Description

Multiple Node.js applications (realtime-api, discordBot, twitchBot) follow nearly identical initialization patterns with minor variations.

Duplicated Code Examples

Common Structure Across Services: 

// Pattern found in: discordBot/src/index.ts, twitchBot/src/index.ts
import { init as initLogger } from './services/logger';
import IORedis from 'ioredis';
import config from './config.json';
import { createServer } from './api/server';
import dotenv from 'dotenv';

type BackendMethod = 'memory' | 'disk' | 'redis';

const initServices = async (method: BackendMethod) => {
  console.log(`Using ${method} as backend`);
  switch (method) {
    case 'redis': {
      const redis = new IORedis({
        host: process.env.REDIS_HOST,
        port: parseInt(process.env.REDIS_PORT || '6379'),
        password: process.env.REDIS_PASSWORD,
        username: process.env.REDIS_USERNAME,
        keyPrefix: 'botname:', // Only difference
      });
      // Identical Redis connection handling
    }
  }
};

const start = async () => {
  initLogger('console', {
    prefix: 'BotName', // Only difference
    level: 'debug',
  });
  const backendMethod = config.USE_REDIS ? 'redis' : config.USE_DISK ? 'disk' : 'memory';
  await initServices(backendMethod);

  const express = createServer();
  express.listen(config.PORT, () => {
    console.log(`Server is running on port ${config.PORT}`);
  });
};

Duplication Severity: High (90%+ identical code) Applications Affected: discordBot, twitchBot, realtime-api Extraction Complexity: Medium

2. Configuration Management Pattern

Pattern Description

Each service maintains its own config.json with overlapping settings and identical structure.

Duplicated Configuration Examples

Common Configuration Structure: 

{
  "PORT": 3000,
  "CLERK_DEVELOPMENT_SECRET_KEY": "sk_test_pUklD9Rf8iWvkrOOj1krL1bVif3oF16zjXfvBNrxlf",
  "USE_REDIS": true,
  "USE_DISK": false,
  "USE_AUTH": true
}

Specific Duplications: - Clerk Secret Key: Identical across all services - Redis Configuration: Same USE_REDIS, USE_DISK flags - Port Configuration: Default port 3000 across services - Authentication Settings: Identical USE_AUTH flags

Duplication Severity: High (70%+ identical configuration) Applications Affected: realtime-api, discordBot, twitchBot Extraction Complexity: Low

3. Build Configuration Pattern

Pattern Description

Webpack configurations are nearly identical across Node.js applications with only path differences.

Duplicated Build Code Examples

Webpack Configuration Pattern: 

// Pattern found in: discordBot, twitchBot, realtime-api
const { NxAppWebpackPlugin } = require('@nx/webpack/app-plugin');
const { join } = require('path');

module.exports = {
  output: {
    path: join(__dirname, '../../../dist/apps/bots/serviceName'), // Only difference
  },
  plugins: [
    new NxAppWebpackPlugin({
      target: 'node',
      compiler: 'tsc',
      main: './src/index.ts',
      tsConfig: './tsconfig.app.json',
      assets: ['./src/assets'], // Sometimes empty array
      optimization: false,
      outputHashing: 'none',
      sourceMap: true,
    }),
  ],
};

Duplication Severity: High (95%+ identical code) Applications Affected: All Node.js services Extraction Complexity: Low

4. TypeScript Configuration Pattern

Pattern Description

TypeScript configurations are duplicated across applications with minimal variations.

Duplicated Configuration Examples

Common tsconfig.app.json Structure: - Identical compiler options - Same include/exclude patterns - Consistent path mappings - Only differences in extends paths

Duplication Severity: Medium (80%+ identical configuration) Applications Affected: All TypeScript applications Extraction Complexity: Low

5. Docker Configuration Pattern

Pattern Description

Dockerfile configurations follow identical patterns across containerized services.

Duplicated Docker Examples

Common Dockerfile Structure: 

# Pattern found in: realtime-api, discordBot, twitchBot
FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD ["npm", "start"]

Duplication Severity: High (90%+ identical) Applications Affected: realtime-api, discordBot, twitchBot Extraction Complexity: Low

6. Express Server Setup Pattern

Pattern Description

Express server creation and middleware setup follows identical patterns across services.

Duplicated Server Code Examples

Common Express Setup Pattern: 

// Pattern inferred from api/server.ts files
import express from 'express';
import cors from 'cors';
import { errorHandler } from './middleware/errorHandler';
import { authMiddleware } from './middleware/authMiddleware';

export const createServer = () => {
  const app = express();

  app.use(cors());
  app.use(express.json());
  app.use(authMiddleware);

  // Route setup
  app.use('/api', routes);

  app.use(errorHandler);

  return app;
};

Duplication Severity: High (85%+ identical code) Applications Affected: All Express-based services Extraction Complexity: Medium

7. Redis Connection Management Pattern

Pattern Description

Redis connection setup, error handling, and reconnection logic is duplicated across services.

Duplicated Redis Code Examples

Common Redis Setup: 

const redis = new IORedis({
  host: process.env.REDIS_HOST,
  port: parseInt(process.env.REDIS_PORT || '6379'),
  password: process.env.REDIS_PASSWORD,
  username: process.env.REDIS_USERNAME,
  keyPrefix: 'service:', // Only difference
});

redis.on('connect', () => {
  console.log('Connected to Redis');
});

redis.on('error', (err) => {
  console.error('Redis error:', err);
  redis.connect();
});

redis.on('reconnecting', () => {
  console.log('Reconnecting to Redis');
});

Duplication Severity: High (95%+ identical code) Applications Affected: All Redis-using services Extraction Complexity: Low

8. Logging Configuration Pattern

Pattern Description

Logger initialization follows identical patterns with only prefix differences.

Duplicated Logging Code Examples

Common Logger Setup: 

initLogger('console', {
  prefix: 'ServiceName', // Only difference
  level: 'debug',
  // filePath: `logs/service-${new Date().toISOString().replace(/:/g, '-')}.log`,
  // console: true,
});

Duplication Severity: High (90%+ identical code) Applications Affected: All services with logging Extraction Complexity: Low

9. Environment Variable Handling Pattern

Pattern Description

Environment variable loading and validation follows identical patterns.

Duplicated Environment Code Examples

Common dotenv Setup: 

import dotenv from 'dotenv';
dotenv.config();

// Environment variable access patterns
process.env.REDIS_HOST
process.env.REDIS_PORT || '6379'
process.env.REDIS_PASSWORD
process.env.REDIS_USERNAME

Duplication Severity: Medium (70%+ identical patterns) Applications Affected: All Node.js services Extraction Complexity: Low

10. AWS S3 URL Pattern

Pattern Description

S3 URL construction is duplicated across multiple pipeline and build scripts.

Duplicated S3 Code Examples

Common S3 URL Pattern: 

// Pattern found in multiple pipeline files
const bucketURL = `https://${BUCKET_NAME}.s3.amazonaws.com`;
const resultUrl = `https://${BUCKET_NAME}.s3.amazonaws.com/${s3Path}`;

Duplication Severity: Medium (60%+ identical patterns) Applications Affected: All pipeline and build scripts Extraction Complexity: Low

Frontend Duplication Patterns

11. Vite Configuration Pattern

Pattern Description

Vite configurations share common plugins and settings across React applications.

Duplicated Vite Code Examples

Common Vite Setup: 

import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
import { nxViteTsPaths } from '@nx/vite/plugins/nx-tsconfig-paths.plugin';
import { sentryVitePlugin } from '@sentry/vite-plugin';

export default defineConfig({
  plugins: [
    react(),
    nxViteTsPaths(),
    sentryVitePlugin({
      org: config.SENTRY_ORG,
      project: config.SENTRY_PROJECT,
      authToken: process.env.SENTRY_AUTH_TOKEN,
    }),
  ],
  build: {
    outDir: '../../dist/apps/appname',
    reportCompressedSize: true,
    commonjsOptions: {
      transformMixedEsModules: true,
    },
    sourcemap: true,
  },
});

Duplication Severity: High (80%+ identical configuration) Applications Affected: All Vite-based React applications Extraction Complexity: Medium

12. React Component Patterns

Pattern Description

Common React patterns like authentication hooks, state management, and error handling are repeated.

Duplicated React Code Examples

Common Authentication Pattern: 

// Pattern inferred from App.tsx
import { useAuth, useSession, useUser } from '@clerk/clerk-react';

const { isSignedIn, isLoaded } = useAuth();
const { session } = useSession();
const { user } = useUser();

// Token refresh logic
const updateProfile = useCallback(async () => {
  if (isSignedIn && user && isLoaded) {
    const token = await session?.getToken({ template: 'host-token' });
    // Token processing logic
  }
}, [isSignedIn, user, isLoaded, session]);

Duplication Severity: Medium (60%+ similar patterns) Applications Affected: All React applications with authentication Extraction Complexity: High

Library Usage Duplication

13. Import Statement Patterns

Pattern Description

Similar import patterns are repeated across applications using the same libraries.

Duplicated Import Examples

Common Library Imports: 

// FontAwesome imports
import { FontAwesomeIcon } from '@fortawesome/react-fontawesome';
import { faIcon1, faIcon2 } from '@fortawesome/pro-solid-svg-icons';

// Shadcn imports
import { Button, Input, Dialog } from '@playcastdotio/Shadcn';

// State management imports
import { useState, useEffect, useContext } from 'react';

Duplication Severity: Low (40%+ similar patterns) Applications Affected: All React applications Extraction Complexity: High

Duplication Impact Analysis

Maintenance Overhead

  1. Configuration Changes: Require updates across multiple files
  2. Dependency Updates: Must be synchronized across services
  3. Bug Fixes: Need to be applied in multiple locations
  4. Feature Additions: Require implementation in multiple places

Consistency Issues

  1. Version Mismatches: Different services may use different library versions
  2. Configuration Drift: Settings may become inconsistent over time
  3. Implementation Variations: Similar functionality implemented differently

Development Velocity Impact

  1. Slower Feature Development: Need to implement changes multiple times
  2. Increased Testing Overhead: Similar code needs testing in multiple contexts
  3. Higher Onboarding Complexity: Developers need to understand multiple similar implementations

Extraction Priority Matrix

High Priority (High Impact, Low Complexity)

  1. Configuration Management: Centralized config system
  2. Build Configurations: Shared Webpack/Vite configurations
  3. Redis Connection Management: Shared Redis utility
  4. Logging Setup: Centralized logging configuration
  5. Docker Configurations: Base Docker images and configurations

Medium Priority (High Impact, Medium Complexity)

  1. Backend Service Initialization: Shared service bootstrap library
  2. Express Server Setup: Common Express application factory
  3. Vite Configuration: Shared Vite configuration presets
  4. Environment Variable Handling: Centralized environment management

Lower Priority (Medium Impact, High Complexity)

  1. React Authentication Patterns: Shared authentication hooks
  2. Component Patterns: Common React component abstractions
  3. State Management Patterns: Shared state management utilities

Phase 1: Infrastructure Duplication (Weeks 1-2)

  • Extract configuration management system
  • Create shared build configuration templates
  • Implement centralized logging utility
  • Standardize Docker configurations

Phase 2: Service Duplication (Weeks 3-4)

  • Create shared service initialization library
  • Extract Redis connection management
  • Implement common Express server factory
  • Standardize environment variable handling

Phase 3: Frontend Duplication (Weeks 5-6)

  • Create shared Vite configuration presets
  • Extract common React authentication patterns
  • Implement shared component utilities
  • Standardize import patterns

Phase 4: Advanced Patterns (Weeks 7-8)

  • Create advanced state management abstractions
  • Implement shared testing utilities
  • Extract complex business logic patterns
  • Optimize and refine extracted libraries

Success Metrics

Quantitative Metrics

  • Lines of Code Reduction: Target 20-30% reduction in duplicated code
  • File Count Reduction: Reduce configuration files by 50%+
  • Dependency Consolidation: Reduce unique dependency versions by 30%+

Qualitative Metrics

  • Developer Experience: Faster onboarding and development
  • Consistency: Uniform patterns across applications
  • Maintainability: Easier platform-wide changes
  • Testing: Reduced test duplication and improved coverage