mingsheng.li/tool-databridge
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
main
tool-databridge/README.md

852 B
Raw Permalink Blame History

Databridge - Data Pipeline System

Databridge is a flexible data pipeline system for processing and transferring data between various sources and destinations. It is designed to run on Kubernetes and supports multiple data processing pipelines.

Features

  • DBF to PostgreSQL: Import data from DBF files to PostgreSQL
  • CSV Export: Export data from PostgreSQL to CSV files
  • Kubernetes Native: Designed to run as Kubernetes Jobs
  • ZFS Storage: Supports ZFS persistent storage
  • Parameterized Pipelines: Flexible configuration via environment variables

Getting Started

Prerequisites

  • Kubernetes cluster
  • ZFS storage provisioner
  • PostgreSQL database

Installation

  1. Deploy Storage Infrastructure: 
    kubectl apply -f k8s/pv.yaml
kubectl apply -f k8s/pvc.yaml
kubectl apply -f k8s/rbac.yaml