Monitoring & Logging

Learn how to monitor your CronJS job executions, view detailed logs, and track performance metrics.

Overview

CronJS provides comprehensive monitoring and logging capabilities to help you track job performance, debug issues, and ensure your automation runs smoothly.

Execution Monitoring

Real-Time Job Monitoring

CronJS offers real-time monitoring through the web interface:

What’s Monitored:

• Job execution status - Currently running, completed, or failed
• Execution start times - When jobs begin running
• Job queue status - Jobs waiting to execute
• Resource utilization - CPU and memory usage during execution

Note: Real-time logs are not currently available, but execution status is updated in real-time.

Job Status Indicators

Your dashboard shows various status indicators:

• 🟢 Success - Job completed without errors (exit code 0)
• 🔴 Failed - Job completed with errors (non-zero exit code)
• 🟡 Running - Job is currently executing
• ⏸️ Disabled - Job is disabled and won’t run on schedule
• ⏰ Scheduled - Job is enabled and waiting for next execution

Execution History

What’s Captured

Each job execution automatically records:

• Start time - Exact timestamp when execution began
• End time - When the job completed or was terminated
• Exit code - Process exit code (0 = success, non-zero = error)
• Console output - All stdout logs from your JavaScript code
• Error messages - stderr output and exception details
• Execution duration - Total runtime in milliseconds

Accessing Execution History

1. Navigate to job details page from your dashboard
2. Scroll to execution history section
3. View recent executions in reverse chronological order
4. Click on specific execution to view detailed logs

Execution History Example

// Your job code automatically generates execution records
console.log("Job started at:", new Date().toISOString());

try {
  // Job logic here
  const result = await performTask();
  console.log("Task completed successfully:", result);
} catch (error) {
  console.error("Task failed:", error.message);
  throw error; // This will result in non-zero exit code
}

console.log("Job finished at:", new Date().toISOString());

Generated Execution Record:

• Start time: 2024-12-19T10:00:00.123Z
• End time: 2024-12-19T10:00:15.456Z
• Duration: 15,333ms
• Exit code: 0 (success)
• Logs: All console.log and console.error output

Log Management

Log Retention

Current Retention Policy:

• Storage location - MongoDB database
• Retention period - Not explicitly configured (stored indefinitely)
• Future enhancement - Configurable retention policies planned

Access Control:

• Only job owners can view their execution logs
• Complete isolation between user accounts
• No cross-user log access

Log Format

Logs are captured in chronological order with timestamps:

2024-12-19T10:00:00.123Z - Job started at: 2024-12-19T10:00:00.123Z
2024-12-19T10:00:02.456Z - Fetching data from API...
2024-12-19T10:00:05.789Z - Data received: 142 records
2024-12-19T10:00:10.012Z - Processing complete
2024-12-19T10:00:15.345Z - Job finished at: 2024-12-19T10:00:15.345Z

Best Logging Practices

Structured Logging

// ✅ Good logging practices
console.log("Starting data sync job");
console.log("API URL:", process.env.API_URL ? "configured" : "missing");
console.log("Processing", records.length, "records");

// Include timestamps in your logs
const timestamp = new Date().toISOString();
console.log(`[${timestamp}] Processing batch ${batchNumber}`);

// Log progress for long-running jobs
for (let i = 0; i < items.length; i++) {
  if (i % 100 === 0) {
    console.log(`Progress: ${i}/${items.length} items processed`);
  }
}

Error Logging

// ✅ Detailed error logging
try {
  const response = await fetch(apiUrl);
  if (!response.ok) {
    throw new Error(`HTTP ${response.status}: ${response.statusText}`);
  }
} catch (error) {
  console.error("API call failed:");
  console.error("Error message:", error.message);
  console.error("Stack trace:", error.stack);
  console.error("API URL:", apiUrl);
  throw error;
}

Avoid Logging Sensitive Data

// ❌ Don't log sensitive information
console.log("API Key:", process.env.API_KEY); // NEVER do this

// ✅ Log safely
console.log("API Key configured:", !!process.env.API_KEY);
console.log("Database connection:", dbUrl ? "configured" : "missing");

// ✅ Sanitize data before logging
const sanitizedUser = {
  id: user.id,
  email: user.email.replace(/(.{2}).*@/, "$1***@"),
  role: user.role,
};
console.log("Processing user:", JSON.stringify(sanitizedUser));

Performance Monitoring

Execution Time Tracking

Monitor how long your jobs take to run:

// Track execution time
const startTime = Date.now();

// Your job logic
await performTasks();

const endTime = Date.now();
const duration = endTime - startTime;

console.log(
  `Job completed in ${duration}ms (${(duration / 1000).toFixed(2)}s)`
);

// Alert on slow execution
if (duration > 30000) {
  // 60 seconds
  console.warn("Job took longer than expected");
}

Resource Usage Monitoring

Track memory and CPU usage:

// Monitor memory usage
function logMemoryUsage(label = "") {
  const usage = process.memoryUsage();
  console.log(`Memory ${label}:`, {
    heapUsed: `${Math.round(usage.heapUsed / 1024 / 1024)}MB`,
    heapTotal: `${Math.round(usage.heapTotal / 1024 / 1024)}MB`,
    external: `${Math.round(usage.external / 1024 / 1024)}MB`,
  });
}

logMemoryUsage("at start");

// Your resource-intensive task
await processLargeDataset();

logMemoryUsage("after processing");

Performance Benchmarking

Track performance over time:

// Performance benchmarking
const metrics = {
  startTime: Date.now(),
  recordsProcessed: 0,
  apiCalls: 0,
  errors: 0,
};

try {
  for (const record of records) {
    const startRecord = Date.now();

    await processRecord(record);

    metrics.recordsProcessed++;
    metrics.apiCalls++;

    const recordTime = Date.now() - startRecord;
    if (recordTime > 1000) {
      console.warn(`Slow record processing: ${recordTime}ms`);
    }
  }
} catch (error) {
  metrics.errors++;
  throw error;
} finally {
  // Log final metrics
  const totalTime = Date.now() - metrics.startTime;
  console.log("Performance Summary:", {
    totalTime: `${totalTime}ms`,
    recordsProcessed: metrics.recordsProcessed,
    avgTimePerRecord: `${Math.round(totalTime / metrics.recordsProcessed)}ms`,
    apiCalls: metrics.apiCalls,
    errors: metrics.errors,
  });
}

Debugging Failed Jobs

Common Failure Patterns

Timeout Errors

// Handle timeout scenarios
const timeout = 50000; // 50 seconds (under 60s limit)
const controller = new AbortController();

setTimeout(() => controller.abort(), timeout);

try {
  const response = await fetch(apiUrl, {
    signal: controller.signal,
  });
} catch (error) {
  if (error.name === "AbortError") {
    console.error("Request timed out after", timeout, "ms");
  }
  throw error;
}

Memory Errors

// Process data in chunks to avoid memory issues
async function processInChunks(data, chunkSize = 100) {
  console.log(`Processing ${data.length} items in chunks of ${chunkSize}`);

  for (let i = 0; i < data.length; i += chunkSize) {
    const chunk = data.slice(i, i + chunkSize);
    console.log(
      `Processing chunk ${Math.floor(i / chunkSize) + 1}/${Math.ceil(
        data.length / chunkSize
      )}`
    );

    await processChunk(chunk);

    // Force garbage collection opportunity
    if (global.gc) {
      global.gc();
    }
  }
}

Network Errors

// Robust network error handling
async function robustFetch(url, options = {}, maxRetries = 3) {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      console.log(`Attempt ${attempt} to fetch: ${url}`);

      const response = await fetch(url, options);

      if (response.ok) {
        console.log("Request successful");
        return response;
      }

      throw new Error(`HTTP ${response.status}: ${response.statusText}`);
    } catch (error) {
      console.error(`Attempt ${attempt} failed:`, error.message);

      if (attempt === maxRetries) {
        throw new Error(`All ${maxRetries} attempts failed: ${error.message}`);
      }

      // Exponential backoff
      const delay = 1000 * Math.pow(2, attempt - 1);
      console.log(`Waiting ${delay}ms before retry...`);
      await new Promise((resolve) => setTimeout(resolve, delay));
    }
  }
}

Debugging Checklist

When a job fails, check:

1. Exit code - What type of error occurred?

   • 0: Success
   • 1: General error
   • 2: Syntax error
   • 130: Terminated by timeout

2. Error messages - Look for specific error details in logs

3. Execution time - Did the job timeout?

4. Memory usage - Did the job exceed memory limits?

5. Environment variables - Are all required variables set?

6. Network connectivity - Are external APIs accessible?

Monitoring Best Practices

Job Health Monitoring

// Health check pattern
async function performHealthCheck() {
  const health = {
    timestamp: new Date().toISOString(),
    status: "healthy",
    checks: {},
  };

  try {
    // Check database connectivity
    health.checks.database = await checkDatabase();

    // Check external API
    health.checks.externalApi = await checkExternalApi();

    // Check required environment variables
    health.checks.environment = checkEnvironment();

    console.log("Health check passed:", JSON.stringify(health));
  } catch (error) {
    health.status = "unhealthy";
    health.error = error.message;
    console.error("Health check failed:", JSON.stringify(health));
    throw error;
  }
}

Alerting Patterns

// Simple alerting pattern
async function sendAlert(message, severity = "warning") {
  console.error(`ALERT [${severity.toUpperCase()}]: ${message}`);

  // Send to external monitoring service
  if (process.env.WEBHOOK_URL) {
    try {
      await fetch(process.env.WEBHOOK_URL, {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({
          text: `CronJS Alert: ${message}`,
          severity,
          timestamp: new Date().toISOString(),
        }),
      });
    } catch (error) {
      console.error("Failed to send alert:", error.message);
    }
  }
}

// Usage
if (errorCount > threshold) {
  await sendAlert(`High error rate detected: ${errorCount} errors`);
}

Monitoring Dashboards

For external monitoring, consider logging to services like:

// Log to external monitoring service
async function logMetrics(metrics) {
  const payload = {
    service: "cronjs",
    job: "data-sync",
    timestamp: new Date().toISOString(),
    metrics,
  };

  if (process.env.MONITORING_ENDPOINT) {
    try {
      await fetch(process.env.MONITORING_ENDPOINT, {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify(payload),
      });
    } catch (error) {
      console.warn("Failed to send metrics:", error.message);
    }
  }
}

// Example usage
await logMetrics({
  execution_time: duration,
  records_processed: recordCount,
  success_rate: successRate,
  memory_used: memoryUsage,
});

Future Monitoring Enhancements

Planned features for upcoming releases:

Advanced Monitoring

• Real-time log streaming - Live log viewing during execution
• Metric dashboards - Visual performance tracking
• Alert configurations - Custom alert rules and notifications
• Performance trends - Historical performance analysis

Integration Features

• Webhook notifications - External alerting integrations
• Monitoring service connectors - DataDog, New Relic, etc.
• Custom metrics - User-defined performance indicators
• Log export - Download logs in various formats

Analytics

• Execution analytics - Success rates, performance trends
• Resource optimization - Recommendations for better performance
• Cost tracking - Resource usage and billing analytics

Next Steps

• Troubleshooting Guide - Solve common monitoring issues
• Job Management - Optimize job configuration
• Advanced Scheduling - Improve job timing
• Getting Started - Create your first monitored job

Master monitoring to build reliable, observable automation with CronJS! 📊