Here are five real-world project-based exercises to master concurrency and parallel programming in C#. Each project comes with an overview, objectives, and hands-on tasks.

🚀 Project 1: High-Performance Web Scraper (Async & Parallel)

Overview

Build a multi-threaded web scraper that fetches data from multiple websites in parallel.

Objectives

✅ Use HttpClient with async/await

✅ Implement Parallel.ForEachAsync() for faster data retrieval

✅ Use ConcurrentBag to store results safely

Implementation Steps

1️⃣ Create a list of URLs to scrape.

2️⃣ Fetch data asynchronously using HttpClient.

3️⃣ Use Parallel.ForEachAsync() for concurrent requests.

4️⃣ Store results in ConcurrentBag.

Code Example 

using System;
using System.Collections.Concurrent;
using System.Net.Http;
using System.Threading.Tasks;

class Program
{
    static async Task Main()
    {
        string[] urls = {
            "https://jsonplaceholder.typicode.com/posts/1",
            "https://jsonplaceholder.typicode.com/posts/2",
            "https://jsonplaceholder.typicode.com/posts/3"
        };

        ConcurrentBag<string> results = new ConcurrentBag<string>();

        await Parallel.ForEachAsync(urls, async (url, _) =>
        {
            using HttpClient client = new HttpClient();
            string content = await client.GetStringAsync(url);
            results.Add(content);
            Console.WriteLine($"Fetched data from {url}");
        });

        Console.WriteLine($"Total responses: {results.Count}");
    }
}

🔹 Enhancement: Store the results in a database or cache (Redis).

🚀 Project 2: Multi-Threaded File Processor

Overview

Process multiple large files concurrently.

Objectives

✅ Read multiple files asynchronously

✅ Use Parallel.ForEach() to process files in parallel

✅ Implement progress tracking

Implementation Steps

1️⃣ Read file names from a directory.

2️⃣ Use Parallel.ForEach() to process files.

3️⃣ Store results in a thread-safe collection.

Code Example 

using System;
using System.Collections.Concurrent;
using System.IO;
using System.Threading.Tasks;

class Program
{
    static void ProcessFile(string filePath)
    {
        string content = File.ReadAllText(filePath);
        Console.WriteLine($"Processed: {filePath} (Length: {content.Length})");
    }

    static void Main()
    {
        string[] files = Directory.GetFiles("C:\\Logs", "*.log");

        Parallel.ForEach(files, ProcessFile);

        Console.WriteLine("All files processed.");
    }
}

🔹 Enhancement: Save processed data in a database or cloud storage.

🚀 Project 3: Concurrent Order Processing System (Producer-Consumer)

Overview

Simulate a high-performance order processing system.

Objectives

✅ Implement the Producer-Consumer pattern

✅ Use BlockingCollection for safe multi-threaded queueing

✅ Optimize performance with ThreadPool

Implementation Steps

1️⃣ The Producer generates orders and adds them to a queue.

2️⃣ The Consumer processes orders concurrently.

3️⃣ Use BlockingCollection for thread-safe queueing.

Code Example 

using System;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;

class Program
{
    static BlockingCollection<string> orderQueue = new BlockingCollection<string>();

    static void OrderProducer()
    {
        for (int i = 1; i <= 10; i++)
        {
            orderQueue.Add($"Order-{i}");
            Console.WriteLine($"Added Order-{i}");
            Thread.Sleep(100);
        }
        orderQueue.CompleteAdding();
    }

    static void OrderConsumer()
    {
        foreach (var order in orderQueue.GetConsumingEnumerable())
        {
            Console.WriteLine($"Processing {order}");
            Thread.Sleep(200);
        }
    }

    static void Main()
    {
        Task.Run(OrderProducer);
        Task.Run(OrderConsumer).Wait();
    }
}

🔹 Enhancement: Implement Kafka or RabbitMQ for real-world distributed order processing.

🚀 Project 4: Parallel Image Processing System

Overview

Apply image processing filters (resize, grayscale, blur) in parallel.

Objectives

✅ Load multiple images from a folder

✅ Use Parallel.ForEach() to apply filters

✅ Save processed images

Implementation Steps

1️⃣ Read images from a directory.

2️⃣ Apply image filters using Parallel.ForEach().

3️⃣ Save processed images to a new folder.

Code Example 

using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using System.Threading.Tasks;

class Program
{
    static void ProcessImage(string filePath)
    {
        using Bitmap image = new Bitmap(filePath);
        string outputPath = Path.Combine("ProcessedImages", Path.GetFileName(filePath));

        // Convert to grayscale
        for (int x = 0; x < image.Width; x++)
        {
            for (int y = 0; y < image.Height; y++)
            {
                Color pixel = image.GetPixel(x, y);
                int gray = (pixel.R + pixel.G + pixel.B) / 3;
                image.SetPixel(x, y, Color.FromArgb(gray, gray, gray));
            }
        }

        image.Save(outputPath, ImageFormat.Jpeg);
        Console.WriteLine($"Processed: {filePath}");
    }

    static void Main()
    {
        string[] files = Directory.GetFiles("C:\\Images", "*.jpg");

        Parallel.ForEach(files, ProcessImage);

        Console.WriteLine("All images processed.");
    }
}

🔹 Enhancement: Use GPU acceleration (e.g., OpenCV) for faster processing.

🚀 Project 5: Scalable Background Job System for ASP.NET Core

Overview

Build a background task processing system using Hangfire or Worker Services.

Objectives

✅ Schedule background jobs

✅ Process jobs asynchronously

✅ Use dependency injection for services

Implementation Steps

1️⃣ Create a Worker Service in ASP.NET Core.

2️⃣ Use Task.Run() for job execution.

3️⃣ Store job results in a database.

Code Example 

using Microsoft.Extensions.Hosting;
using System;
using System.Threading;
using System.Threading.Tasks;

public class BackgroundJob : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            Console.WriteLine($"Processing job at {DateTime.Now}");
            await Task.Delay(5000, stoppingToken);
        }
    }
}

class Program
{
    static async Task Main()
    {
        await Host.CreateDefaultBuilder()
            .ConfigureServices((_, services) => services.AddHostedService<BackgroundJob>())
            .RunConsoleAsync();
    }
}

🔹 Enhancement: Integrate Redis or RabbitMQ for distributed job queues.

🔥 Summary Table of Projects

Project Key Concepts Technologies
Web Scraper Async/Await, Parallel Requests HttpClient, ConcurrentBag
File Processor Parallel.ForEach, File I/O Parallel Programming
Order Processing Producer-Consumer, Thread Safety BlockingCollection, ThreadPool
Image Processing Parallel Image Manipulation Bitmap, Parallel.ForEach
Background Jobs Scalable Task Processing ASP.NET Worker Service