# Article 2: Installing Kestra: Your First Steps to Powerful Orchestration.

## **Complete Setup Guide for Local, Docker, and Cloud Deployments.**

### **Introduction: Why Installation Matters**

Think of Kestra as the conductor of your data orchestra. Just as a great conductor needs the right stage setup, Kestra needs proper installation to perform at its best. In this guide, we'll walk through every installation option—from a quick local setup to enterprise-grade deployments.

**Quick Decision Guide:**

* **Just exploring?** → Use Docker standalone (5 minutes)
    
* **Developing workflows?** → Use Docker Compose (10 minutes)
    
* **Production deployment?** → Use Kubernetes/Helm (30 minutes)
    
* **Enterprise scale?** → Use Terraform on cloud (60 minutes)
    

---

## **Option 1: Local Development (Fastest Path)**

### **Prerequisites Checklist**

Before we begin, ensure you have:

* ✅ Docker installed ([Get Docker](https://docs.docker.com/get-docker/))
    
* ✅ 2GB free RAM
    
* ✅ 10GB free disk space
    
* ✅ Internet connection
    

### **Method A: Quick Start with Docker (5 Minutes)**

```bash
# Run this single command
docker run --rm \
  -p 8080:8080 \
  -v $(pwd)/flows:/app/flows \
  -v $(pwd)/storage:/app/storage \
  kestra/kestra:latest \
  standalone

# Or for Windows PowerShell
docker run --rm `
  -p 8080:8080 `
  -v ${PWD}/flows:/app/flows `
  -v ${PWD}/storage:/app/storage `
  kestra/kestra:latest `
  standalone
```

**What this does:**

* Starts Kestra with embedded databases
    
* Maps local directories for flows and storage
    
* Opens port 8080 for the web UI
    
* Uses temporary containers (removed on stop)
    

**Access the UI:**

1. Open [http://localhost:8080](http://localhost:8080/)
    
2. Default credentials: `admin` / `password` (change immediately!)
    

### **Method B: Docker Compose for Development (Recommended)**

Create a `docker-compose.yml` file:

```bash
version: '3.8'

services:
  kestra:
    image: kestra/kestra:latest
    container_name: kestra-server
    ports:
      - "8080:8080"
      - "8081:8081"  # For metrics if needed
    environment:
      - KESTRA_CONFIGURATION= |
          kestra:
            server:
              basic-auth:
                enabled: true
                username: admin
                password: ${KESTRA_PASSWORD:-changeme}
            repository:
              type: postgres
            storage:
              type: local
            queue:
              type: postgres
    volumes:
      - ./flows:/app/flows      # Your workflow definitions
      - ./plugins:/app/plugins  # Custom plugins
      - ./logs:/app/logs        # Execution logs
      - ./storage:/app/storage  # File storage
    depends_on:
      postgres:
        condition: service_healthy
    restart: unless-stopped

  postgres:
    image: postgres:15-alpine
    container_name: kestra-postgres
    environment:
      POSTGRES_DB: kestra
      POSTGRES_USER: kestra
      POSTGRES_PASSWORD: kestra
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./init-db:/docker-entrypoint-initdb.d  # Optional: initial scripts
    ports:
      - "5432:5432"  # Optional: expose for external tools
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U kestra"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

volumes:
  postgres_data:
```

**Start everything with one command:**

```bash
# Create necessary directories
mkdir -p flows plugins logs storage init-db

# Start all services
docker-compose up -d

# Check status
docker-compose ps

# View logs
docker-compose logs -f kestra
```

**What you get:**

* ✅ Persistent database (PostgreSQL)
    
* ✅ File storage that survives restarts
    
* ✅ Proper service health checks
    
* ✅ Easy log viewing
    
* ✅ Production-like setup locally
    

---

## **Option 2: Production-Ready Kubernetes Setup**

### **Prerequisites:**

* Kubernetes cluster (Minikube, Docker Desktop, EKS, AKS, GKE)
    
* Helm 3+
    
* kubectl configured
    

### **Step 1: Create Namespace and Secrets**

```bash
# Create namespace
kubectl create namespace kestra

# Create a secret for credentials
kubectl create secret generic kestra-secrets \
  --namespace kestra \
  --from-literal=postgres-password='YourSecurePassword123!' \
  --from-literal=admin-password='AnotherSecurePassword456!'

# Create storage class (if needed)
cat <<EOF | kubectl apply -f -
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: kestra-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
EOF
```

### **Step 2: Deploy with Helm**

```bash
# Add Kestra Helm repository
helm repo add kestra https://kestra.io/helm
helm repo update

# Install Kestra with custom values
cat > kestra-values.yaml << EOF
# kestra-values.yaml
global:
  storageClass: kestra-storage

server:
  replicaCount: 2
  autoscaling:
    enabled: true
    minReplicas: 2
    maxReplicas: 10
  resources:
    requests:
      memory: "1Gi"
      cpu: "500m"
    limits:
      memory: "2Gi"
      cpu: "1000m"
  extraEnv:
    - name: JAVA_OPTS
      value: "-Xmx1g -Xms512m"

postgresql:
  enabled: true
  auth:
    database: kestra
    username: kestra
    existingSecret: kestra-secrets
    secretKeys:
      adminPasswordKey: postgres-password
  persistence:
    size: 50Gi
    storageClass: kestra-storage

ingress:
  enabled: true
  className: nginx
  hosts:
    - host: kestra.yourcompany.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: kestra-tls
      hosts:
        - kestra.yourcompany.com

redis:
  enabled: true
  architecture: standalone
  auth:
    existingSecret: kestra-secrets
    secretKeys:
      redis-passwordKey: redis-password

elasticsearch:
  enabled: true  # For enhanced logging and search
EOF

# Install the chart
helm install kestra kestra/kestra \
  --namespace kestra \
  --values kestra-values.yaml \
  --wait

# Check deployment status
kubectl get all -n kestra
```

### **Step 3: Configure Ingress and TLS**

```bash
# If using cert-manager for TLS
cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: kestra-tls
  namespace: kestra
spec:
  secretName: kestra-tls
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
    - kestra.yourcompany.com
EOF
```

### **Step 4: Verify Deployment**

```bash
# Check pods
kubectl get pods -n kestra -w

# Check services
kubectl get svc -n kestra

# View logs
kubectl logs -n kestra deployment/kestra-server -f

# Port-forward for local access (if needed)
kubectl port-forward -n kestra svc/kestra-server 8080:80
```

---

## **Option 3: Cloud-Specific Deployments**

### **AWS ECS Setup (Fargate)**

```bash
# ecs-task-definition.json
{
  "family": "kestra-server",
  "networkMode": "awsvpc",
  "taskRoleArn": "arn:aws:iam::123456789012:role/ecsTaskRole",
  "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole",
  "cpu": "2048",
  "memory": "4096",
  "containerDefinitions": [
    {
      "name": "kestra",
      "image": "kestra/kestra:latest",
      "portMappings": [
        {
          "containerPort": 8080,
          "protocol": "tcp"
        }
      ],
      "environment": [
        {
          "name": "KESTRA_CONFIGURATION",
          "value": "kestra.repository.type=postgres\nkestra.storage.type=s3"
        }
      ],
      "secrets": [
        {
          "name": "DB_PASSWORD",
          "valueFrom": "arn:aws:secretsmanager:region:account:secret:db_password"
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/kestra",
          "awslogs-region": "us-east-1",
          "awslogs-stream-prefix": "ecs"
        }
      }
    }
  ]
}
```

**Deploy with AWS CLI:**

```bash
# Register task definition
aws ecs register-task-definition \
  --cli-input-json file://ecs-task-definition.json

# Create service
aws ecs create-service \
  --cluster production \
  --service-name kestra \
  --task-definition kestra-server:1 \
  --desired-count 3 \
  --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[subnet-xxx,subnet-yyy],securityGroups=[sg-zzz],assignPublicIp=ENABLED}"
```

### **Google Cloud Run (Serverless)**

```bash
# Deploy to Cloud Run
gcloud run deploy kestra \
  --image=kestra/kestra:latest \
  --platform=managed \
  --region=us-central1 \
  --allow-unauthenticated \
  --memory=2Gi \
  --cpu=2 \
  --set-env-vars="KESTRA_CONFIGURATION=..."
  
# Set up Cloud SQL connection
gcloud run services update kestra \
  --add-cloudsql-instances=PROJECT:REGION:INSTANCE \
  --set-env-vars="DB_HOST=/cloudsql/PROJECT:REGION:INSTANCE"
```

### **Azure Container Instances**

```bash
# Create container instance
az container create \
  --resource-group kestra-rg \
  --name kestra-server \
  --image kestra/kestra:latest \
  --ports 8080 \
  --memory 4 \
  --cpu 2 \
  --environment-variables \
    KESTRA_CONFIGURATION='kestra.repository.type=postgres' \
  --dns-name-label kestra-unique \
  --azure-file-volume-share-name kestra-data \
  --azure-file-volume-account-name mystorageaccount \
  --azure-file-volume-account-key $STORAGE_KEY
```

---

## **Option 4: Bare Metal / VM Installation**

For environments where containers aren't an option:

### **Step 1: Install Java and Dependencies**

```bash
# Ubuntu/Debian
sudo apt update
sudo apt install -y openjdk-17-jre-headless postgresql-15 redis-server

# CentOS/RHEL
sudo yum install -y java-17-openjdk postgresql15-server redis

# macOS
brew install openjdk@17 postgresql@15 redis
```

### **Step 2: Configure Database**

```bash
# Initialize PostgreSQL
sudo postgresql-setup initdb
sudo systemctl start postgresql
sudo systemctl enable postgresql

# Create Kestra database and user
sudo -u postgres psql << EOF
CREATE DATABASE kestra;
CREATE USER kestra WITH PASSWORD 'secure_password';
GRANT ALL PRIVILEGES ON DATABASE kestra TO kestra;
ALTER DATABASE kestra SET timezone TO 'UTC';
EOF

# Update pg_hba.conf for password auth
echo "host    kestra          kestra          127.0.0.1/32            md5" | sudo tee -a /etc/postgresql/15/main/pg_hba.conf
sudo systemctl restart postgresql
```

### **Step 3: Install Kestra**

```bash
# Download Kestra
wget https://github.com/kestra-io/kestra/releases/latest/download/kestra-standalone.jar -O /opt/kestra/kestra.jar

# Create systemd service
cat << EOF | sudo tee /etc/systemd/system/kestra.service
[Unit]
Description=Kestra Workflow Orchestrator
After=network.target postgresql.service redis.service

[Service]
Type=simple
User=kestra
Group=kestra
WorkingDirectory=/opt/kestra
ExecStart=/usr/bin/java -Xmx2g -jar /opt/kestra/kestra.jar server standalone

# Security hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/var/lib/kestra

[Install]
WantedBy=multi-user.target
EOF

# Create user and directories
sudo useradd -r -s /bin/false kestra
sudo mkdir -p /opt/kestra /var/lib/kestra/{flows,storage,logs,plugins}
sudo chown -R kestra:kestra /opt/kestra /var/lib/kestra

# Start service
sudo systemctl daemon-reload
sudo systemctl enable kestra
sudo systemctl start kestra
sudo systemctl status kestra
```

---

## **Verification and Testing**

### **Health Check**

```bash
# Using curl
curl http://localhost:8080/api/v1/health

# Expected response
{
  "status": "UP",
  "components": {
    "database": {
      "status": "UP"
    },
    "storage": {
      "status": "UP"
    }
  }
}

# Using API
curl -X GET "http://localhost:8080/api/v1/configs" \
  -H "Authorization: Basic $(echo -n 'admin:password' | base64)"
```

### **Create Your First Flow Programmatically**

```python
# test_kestra.py
import requests
import json
import yaml

# Configure
BASE_URL = "http://localhost:8080"
USERNAME = "admin"
PASSWORD = "password"
NAMESPACE = "demo"

# Create namespace
auth = (USERNAME, PASSWORD)
headers = {"Content-Type": "application/json"}

namespace_payload = {
    "namespace": NAMESPACE,
    "description": "Demo namespace"
}

response = requests.post(
    f"{BASE_URL}/api/v1/namespaces",
    json=namespace_payload,
    auth=auth,
    headers=headers
)

# Create a simple flow
flow_yaml = """
id: hello-world
namespace: demo
description: My first Kestra flow

tasks:
  - id: hello
    type: io.kestra.plugin.core.log.Log
    message: "Hello, Kestra! 🎉"
    
  - id: get-time
    type: io.kestra.plugin.core.debug.Return
    format: "Flow started at {{ execution.startDate }}"
    
triggers:
  - id: schedule
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "*/5 * * * *"  # Every 5 minutes
"""

# Create flow
response = requests.post(
    f"{BASE_URL}/api/v1/flows/{NAMESPACE}",
    data=flow_yaml,
    auth=auth,
    headers={"Content-Type": "application/yaml"}
)

# Execute flow
response = requests.post(
    f"{BASE_URL}/api/v1/executions/trigger/{NAMESPACE}/hello-world",
    auth=auth,
    headers=headers
)

print("Flow created and triggered successfully!")
```

### **Test with Kestra CLI**

bash

```bash
# Install CLI
pip install kestra

# Configure
kestra config set-endpoint http://localhost:8080
kestra config set-username admin
kestra config set-password password

# Test commands
kestra namespace list
kestra flow list --namespace demo
kestra flow execute --namespace demo --flow hello-world
kestra execution list --namespace demo
```

---

## **Configuration Deep Dive**

### **Configuration File Structure**

Create `kestra.yml` for custom configurations:

```yaml
# kestra.yml
kestra:
  server:
    basic-auth:
      enabled: true
      username: admin
      password: ${KESTRA_PASSWORD:?Password required}
    
    cors:
      enabled: true
      allowed-origins: ["https://*.yourcompany.com"]
    
    metrics:
      enabled: true
      port: 8081
      path: /metrics
    
  repository:
    type: postgres
    postgres:
      host: ${DB_HOST:localhost}
      port: ${DB_PORT:5432}
      database: kestra
      username: kestra
      password: ${DB_PASSWORD:?DB password required}
      pool-size: 10
      
  storage:
    type: s3  # or local, gcs, azure
    s3:
      region: ${AWS_REGION:us-east-1}
      bucket: kestra-storage
      endpoint: ${AWS_ENDPOINT:}  # For MinIO or other S3-compatible
      access-key-id: ${AWS_ACCESS_KEY_ID:}
      secret-access-key: ${AWS_SECRET_ACCESS_KEY:}
      
  queue:
    type: postgres  # or redis, kafka
    
  cache:
    type: redis
    redis:
      host: ${REDIS_HOST:localhost}
      port: ${REDIS_PORT:6379}
      password: ${REDIS_PASSWORD:}
      
  plugins:
    location: /app/plugins
    repositories:
      - id: kestra-io
        url: https://repo.kestra.io/repository/maven-public/
        
  tasks:
    default-retry:
      type: exponential
      max-attempt: 3
      delay: PT5S
      
  security:
    secret-key: ${SECRET_KEY:?Secret key required for encryption}
    encryption:
      enabled: true
```

### **Environment Variables**

```bash
# .env file example
KESTRA_PASSWORD=SuperSecurePassword123!
DB_HOST=postgres
DB_PORT=5432
DB_PASSWORD=db_password_456
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...
REDIS_HOST=redis
REDIS_PASSWORD=redis_pass_789
SECRET_KEY=$(openssl rand -hex 32)
```

### **Run with Custom Config**

```bash
# Docker with custom config
docker run --rm \
  -p 8080:8080 \
  -v $(pwd)/kestra.yml:/app/kestra.yml \
  -v $(pwd)/.env:/app/.env \
  --env-file .env \
  kestra/kestra:latest \
  server standalone
  
# Or with environment variables
docker run --rm \
  -p 8080:8080 \
  -e KESTRA_PASSWORD=password \
  -e DB_HOST=postgres \
  kestra/kestra:latest \
  server standalone
```

---

## **Troubleshooting Guide**

### **Common Issues and Solutions**

#### **1\. Port Already in Use**

```bash
# Find what's using port 8080
sudo lsof -i :8080

# Kill the process
sudo kill -9 <PID>

# Or change Kestra port
docker run --rm -p 9090:8080 kestra/kestra:latest
```

#### **2\. Database Connection Issues**

```bash
# Test database connection
docker exec -it kestra-postgres psql -U kestra -d kestra

# Reset database (development only)
docker-compose down -v
docker-compose up -d
```

#### **3\. Permission Denied for Storage**

```bash
# Fix volume permissions
sudo chown -R 1000:1000 ./flows ./storage ./logs

# Or run as current user
docker run --rm \
  -u $(id -u):$(id -g) \
  -v $(pwd)/flows:/app/flows \
  kestra/kestra:latest
```

#### **4\. Out of Memory Errors**

```bash
# Increase memory in docker-compose
services:
  kestra:
    deploy:
      resources:
        limits:
          memory: 2G
        reservations:
          memory: 1G
```

#### **5\. Flow Not Appearing**

```bash
# Check if flows are being loaded
curl http://localhost:8080/api/v1/flows

# Check logs for errors
docker-compose logs kestra | grep -i flow

# Verify directory mapping
docker exec -it kestra-server ls -la /app/flows
```

### **Debug Commands**

```bash
# Get detailed system info
curl http://localhost:8080/api/v1/configs/debug

# Check plugin availability
curl http://localhost:8080/api/v1/plugins

# View server metrics
curl http://localhost:8080/api/v1/metrics

# Check queue status
curl http://localhost:8080/api/v1/queues
```

---

## **Post-Installation Checklist**

### **✅ Security Hardening**

```bash
# 1. Change default password
curl -X PUT "http://localhost:8080/api/v1/users/admin" \
  -H "Authorization: Basic $(echo -n 'admin:password' | base64)" \
  -H "Content-Type: application/json" \
  -d '{"password": "NewSecurePassword123!"}'

# 2. Enable HTTPS
# 3. Set up firewall rules
# 4. Configure audit logging
```

### **✅ Backup Strategy**

```bash
# Backup flows
cp -r ./flows ./backups/flows-$(date +%Y%m%d)

# Backup database
docker exec kestra-postgres pg_dump -U kestra kestra > backup-$(date +%Y%m%d).sql

# Backup configuration
cp kestra.yml kestra.yml.backup-$(date +%Y%m%d)
```

### **✅ Monitoring Setup**

```bash
# Install monitoring stack
docker-compose -f monitoring/docker-compose.yml up -d

# Configure Prometheus
echo "
  - job_name: 'kestra'
    static_configs:
      - targets: ['kestra:8081']
" >> prometheus/prometheus.yml
```

---

## **Production Deployment Checklist**

### **Before Going Live:**

* **Performance Testing**: Load test with 100+ concurrent flows
    
* **Disaster Recovery**: Test backup/restore procedures
    
* **High Availability**: Ensure multiple replicas are running
    
* **Monitoring**: Set up alerts for critical metrics
    
* **Security Audit**: Review access controls and encryption
    
* **Documentation**: Document deployment procedures
    
* **Rollback Plan**: Test downgrade procedure
    
* **Load Balancer**: Configure proper load balancing
    
* **DNS Setup**: Configure proper DNS records
    
* **SSL/TLS**: Set up proper certificates
    

### **Critical Metrics to Monitor:**

* Flow execution success rate (&gt;99%)
    
* Average execution time
    
* Queue backlog size
    
* Database connection pool usage
    
* Memory and CPU utilization
    
* Storage usage
    
* API response times
    

---

## **Next Steps: What to Build First**

Now that Kestra is installed, here are some practical projects to start with:

### **Project 1: Data Pipeline Template**

```bash
# Create project structure
mkdir -p flows/{extract,transform,load}
mkdir -p scripts/{python,sql}
mkdir -p config

# Create your first real flow
cat > flows/extract/daily-sales.yml << 'EOF'
id: daily-sales-extract
namespace: sales.pipeline
description: Extract daily sales data

tasks:
  - id: extract-api
    type: io.kestra.plugin.core.http.Download
    uri: "https://api.company.com/sales/{{ execution.startDate | date('yyyy-MM-dd') }}"
    outputFile: "sales_{{ execution.startDate | date('yyyy-MM-dd') }}.json"
EOF
```

### **Project 2: Monitoring Dashboard**

1. Deploy Grafana alongside Kestra
    
2. Import Kestra dashboard templates
    
3. Set up alerts for failed executions
    
4. Create custom metrics dashboards
    

### **Project 3: CI/CD Pipeline**

```bash
# .github/workflows/kestra-deploy.yml
name: Deploy Kestra Flows
on:
  push:
    paths:
      - 'flows/**'
      
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Validate flows
        run: |
          docker run --rm \
            -v $(pwd)/flows:/flows \
            kestra/kestra:latest \
            flow validate /flows
      
      - name: Deploy to Kestra
        run: |
          curl -X POST "https://kestra.company.com/api/v1/flows/batch" \
            -H "Authorization: Bearer ${{ secrets.KESTRA_TOKEN }}" \
            -F "files=@flows/sales-pipeline.yml" \
            -F "namespace=sales"
```

---

## **Community and Support**

### **Getting Help:**

* **Documentation**: [kestra.io/docs](https://kestra.io/docs)
    
* **GitHub Issues**: [github.com/kestra-io/kestra/issues](https://github.com/kestra-io/kestra/issues)
    
* **Community Slack**: [kestra.io/slack](https://kestra.io/slack) (Join 3000+ members)
    
* **Stack Overflow**: Use tag `kestra`
    

### **Contributing:**

```bash
# Clone the repository
git clone https://github.com/kestra-io/kestra.git
cd kestra

# Build from source
./gradlew build

# Run tests
./gradlew test

# Submit pull request
```

---

## **Conclusion**

You now have Kestra running in your chosen environment! Whether you opted for the quick Docker setup or a full Kubernetes deployment, you're ready to start building powerful data workflows.

**Key Takeaways:**

1. **Start simple**: Use Docker standalone for exploration
    
2. **Scale gradually**: Move to Docker Compose, then Kubernetes
    
3. **Configure properly**: Use environment variables and config files
    
4. **Monitor everything**: Set up metrics and alerts from day one
    
5. **Join the community**: The Kestra community is active and helpful
    

### **What's Next?**

In the next article, we'll dive into **Building Your First ETL Pipeline**. We'll cover:

* Extracting data from multiple sources
    
* Transforming with Python and SQL
    
* Loading to databases and data warehouses
    
* Error handling and monitoring
    
* Best practices for production pipelines
    

**Before the next article, try:**

1. Create a namespace for your project
    
2. Build a simple flow that downloads and processes a public dataset
    
3. Explore the web UI and understand the execution view
    
4. Join the Slack community and introduce yourself
    

---

**Remember**: The goal of installation isn't just to get software running—it's to create a foundation that's secure, scalable, and maintainable. Take the time to do it right, and your future self will thank you.

Happy orchestrating! 🚀