27 lines
852 B
Markdown
27 lines
852 B
Markdown
# Databridge - Data Pipeline System
|
|
|
|
Databridge is a flexible data pipeline system for processing and transferring data between various sources and destinations. It is designed to run on Kubernetes and supports multiple data processing pipelines.
|
|
|
|
## Features
|
|
|
|
- **DBF to PostgreSQL**: Import data from DBF files to PostgreSQL
|
|
- **CSV Export**: Export data from PostgreSQL to CSV files
|
|
- **Kubernetes Native**: Designed to run as Kubernetes Jobs
|
|
- **ZFS Storage**: Supports ZFS persistent storage
|
|
- **Parameterized Pipelines**: Flexible configuration via environment variables
|
|
|
|
## Getting Started
|
|
|
|
### Prerequisites
|
|
|
|
- Kubernetes cluster
|
|
- ZFS storage provisioner
|
|
- PostgreSQL database
|
|
|
|
### Installation
|
|
|
|
1. **Deploy Storage Infrastructure**:
|
|
```bash
|
|
kubectl apply -f k8s/pv.yaml
|
|
kubectl apply -f k8s/pvc.yaml
|
|
kubectl apply -f k8s/rbac.yaml |