mingsheng.li/tool-databridge
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9 Commits 1 Branch 0 Tags 172 KiB
Python 76.4%
Shell 21.8%
Dockerfile 1.8%
64dc576ed0
Go to file
mingsheng.li 64dc576ed0
Some checks reported errors
continuous-integration/drone/push Build was killed
Details
修改requirement里面的依赖的版本
2025-07-25 14:07:02 +08:00
k8s 初始合入databridge，用于后续数据的导出导入 2025-07-25 11:54:20 +08:00
scripts 初始合入databridge，用于后续数据的导出导入 2025-07-25 11:54:20 +08:00
src 初始合入databridge，用于后续数据的导出导入 2025-07-25 11:54:20 +08:00
.drone.yml 修改drone文件的配置：恢复构建缓存配置 2025-07-25 13:16:13 +08:00
.gitignore 初始合入databridge，用于后续数据的导出导入 2025-07-25 11:54:20 +08:00
Dockerfile 修改requirement里面的依赖的版本 2025-07-25 13:48:15 +08:00
pyproject.toml 初始合入databridge，用于后续数据的导出导入 2025-07-25 11:54:20 +08:00
README.md 初始合入databridge，用于后续数据的导出导入 2025-07-25 11:54:20 +08:00
requirements.txt 修改requirement里面的依赖的版本 2025-07-25 14:07:02 +08:00

README.md

Databridge - Data Pipeline System

Databridge is a flexible data pipeline system for processing and transferring data between various sources and destinations. It is designed to run on Kubernetes and supports multiple data processing pipelines.

Features

  • DBF to PostgreSQL: Import data from DBF files to PostgreSQL
  • CSV Export: Export data from PostgreSQL to CSV files
  • Kubernetes Native: Designed to run as Kubernetes Jobs
  • ZFS Storage: Supports ZFS persistent storage
  • Parameterized Pipelines: Flexible configuration via environment variables

Getting Started

Prerequisites

  • Kubernetes cluster
  • ZFS storage provisioner
  • PostgreSQL database

Installation

  1. Deploy Storage Infrastructure: 
    kubectl apply -f k8s/pv.yaml
kubectl apply -f k8s/pvc.yaml
kubectl apply -f k8s/rbac.yaml