This is a submission for the Pulumi Deploy and Document Challenge: Get Creative with Pulumi and GitHub
🛠️ What I Built
I created PReQual, an end-to-end pull request quality management system that transforms how teams handle code reviews. By leveraging Pulumi's GitHub provider and Automation API, PReQual automatically configures repositories with standardized branch protection rules while tracking PR activities across organizations.
The heart of the system is a contributor recognition dashboard that visualizes who's creating PRs, who's reviewing them, and where bottlenecks occur. It identifies stale PRs, sends Slack notifications for important events, and provides insights into team collaboration patterns.
What makes PReQual unique is how it combines infrastructure-as-code with workflow analytics. Rather than just deploying infrastructure, I used Pulumi to actively manage GitHub resources (repositories, branch protection, webhooks) while collecting metrics that matter to development teams. The backend runs on an Azure VM provisioned through Pulumi, creating a seamless pipeline from GitHub events to analytics dashboards.
This approach helps teams reward top contributors, balance review workloads, and ensure consistent quality standards across all repositories - turning the often-overlooked PR process into a driver of team improvement.
🔗 Live Demo Link
See the live demo
Note: Due to cloud costs, this demo operates without Azure resources, but fully demonstrates the UI and workflow. The screenshots showcase actual GitHub data collected from test repositories in my organization.
📂 Project Repo
🧭 My Journey
I planned PReQual as a four-phase project that would progressively build from infrastructure to a complete application. What started as an exploration of Pulumi's GitHub capabilities evolved into a comprehensive PR management system.
🏗️ Phase 1: Infrastructure Foundation
Plan: Create Pulumi infrastructure to automatically deploy GitHub repositories with standardized branch protection rules.
Reality: I immediately hit roadblocks with GitHub permissions. Creating repositories within an organization required specific token configurations—I needed to create fine-grained tokens with organization and repository permissions, plus a classic PAT with repo and admin:org access.
When I finally got repository creation working, webhook registration failed because GitHub couldn't reach my Azure VM. Adding admin:hook permissions helped, but then I discovered GitHub doesn't allow multiple webhooks with the same URL in an organization. My Pulumi code was creating new webhook resources with timestamps on each deployment, causing uniqueness conflicts.
VM configuration added another layer of complexity—I initially used dynamic IP addresses, causing webhook failures as the destination URLs kept changing. Switching to static IPs provided the stability needed for webhook delivery.
For credential management, I leveraged Pulumi's config system with the --secret flag to automatically generate and securely store passwords, eliminating manual credential handling across deployments.
📡 Phase 2: Webhook Processing
Plan: Configure the application to receive and process GitHub webhook events, sending notifications to Slack.
Reality: The first deployment failed due to unexpected Python dependency issues in the VM environment. I had to meticulously trace and install missing packages.
My Slack integration initially failed because I hadn't properly enabled incoming webhook permissions in the Slack API settings. I had to navigate Slack's developer portal to toggle the feature to 'On' and generate the correct webhook URL.
Testing webhook delivery required creating test PRs and carefully examining logs to ensure events were being processed correctly—a time-consuming process of trial and error.
💾 Phase 3: Data Management
Plan: Store webhook data in a database to enable analytics and stale PR detection.
Reality: I designed a database structure that seemed perfect on paper, but implementing it revealed gaps in my planning. I needed to create a more comprehensive database layer with specialized components for connection management, models for repositories/users/PRs, analytics for metrics, and a unified handler.
During testing, I discovered my app wasn't capturing comment data correctly because I had specified too narrow a filter. After broadening it to capture all comment types, I hit issues with ODBC driver configuration that prevented database connections.
Deploying with gunicorn in production introduced another set of challenges that weren't apparent in local development.
🖥️ Phase 4: Frontend Development
Plan: Create a Next.js frontend to visualize the PR metrics and provide an intuitive dashboard.
Reality: Building the dashboard components revealed inconsistencies in my backend API responses. Type mismatches between frontend expectations and backend responses created data fetching problems that required refactoring both sides.
Deployment introduced a cascade of issues:
- Package incompatibilities requiring cleanup of .next directories
- Missing dependencies in the VM setup script
- CORS issues blocking frontend-backend communication
- Firewall configuration blocking necessary ports
I had to implement nginx as a reverse proxy rather than directly exposing application ports—a crucial security best practice I hadn't initially considered. Setting up a proper systemd service ensures the backend would automatically restart after VM reboots, addressing reliability concerns.
I never expected that within a week, I could become so comfortable with a new tool like Pulumi. Coming from Terraform and CloudFormation, I was surprised by how quickly I adapted to Pulumi's approach and how powerful the Automation API proved to be. The experience transformed how I think about infrastructure as code—not just as provisioning but as an integral part of application logic.
💡 Beyond Technical Value
- By tracking and visualizing contributions, PReQual creates visibility for team members whose work might otherwise go unnoticed.
- Many teams struggle with uneven review distribution, where a few developers handle most reviews while others rarely participate.
- Organizations often struggle to maintain consistent standards across repositories. PReQual enforces these automatically.
- By measuring not just code production but review activity, PReQual shifts focus toward collaborative improvement.
In essence, PReQual uses infrastructure as code to solve what is fundamentally a human problem: creating a fair, transparent, and quality-focused code review culture where contribution is recognized and rewarded.
🔌 Pulumi GitHub Provider Integration
The core of my project relies on Pulumi's GitHub provider to:
-
Create standardized repositories with consistent settings across an organization. Rather than manually configuring each repository or using GitHub's UI, Pulumi allows me to define repository configurations as code, ensuring that:
- Branch protection rules are consistently applied
- Merge settings follow team standards
- Force pushes are properly restricted
-
Configure organization and repository webhooks that trigger my application when PR events occur. This creates an event-driven architecture where:
- PR creations, reviews, and comments are automatically captured
- My system receives real-time updates about GitHub activity
- Webhook security is properly configured with secrets
🤖 Pulumi Automation API
how I integrated the Pulumi Automation API into my application:
I created a PulumiExecutor class that serves as a bridge between my Flask backend and Pulumi. This allows my application to:
- Deploy infrastructure programmatically based on user inputs
- Create repositories when requested through the UI
- Manage GitHub configurations without direct GitHub API calls
When users configure my application through the setup wizard, Pulumi deploys all necessary infrastructure including:
- Azure VM for webhook processing
- Azure SQL Database for analytics storage
- Network configuration with proper security groups
- GitHub webhook configuration pointing to the VM
🏆 Benefits of This Approach
Using Pulumi this way provided several unique advantages:
- Users can create GitHub resources and Azure infrastructure through a single dashboard without needing Pulumi expertise.
- All repositories follow the same configuration patterns, ensuring consistent quality standards.
- Pulumi's secret management handles GitHub tokens and Azure credentials securely.
- Creating a repository with proper branch protection, webhooks, and database tracking becomes a one-click operation.
This approach demonstrates how Pulumi can enable not just infrastructure deployment, but complete workflow automation that bridges DevOps and application development.
🤝 My Experience with Pulumi Copilot
I am new to Pulumi, so I used Pulumi Copilot to explore all the UI and sometimes i get unexpected exit program in that time it helps me to identify the issue
