Enterprise integration platform

SegmentSync: Enterprise Audience & Data Integration Platform

A reusable integration platform for configuring, scheduling, monitoring and recovering audience and dataset movement across enterprise systems.

Client
Enautics Group, Inc. · Mountain America Credit Union
Role
Solution Architect & Lead Developer
Operating flow
Configure once → schedule → monitor → reuse

01 / The problem

Every new data flow risked becoming another custom pipeline

Each source-to-destination pair repeated mapping, scheduling, retry and monitoring logic. The product question was how every integration could reuse the same control layer.

01BigQuery → Oracle EloquaMapping · schedule · retry · monitoring
02BigQuery → Adobe Experience PlatformMapping · schedule · retry · monitoring
03AEP → Oracle EloquaMapping · schedule · retry · monitoring
04REST API → destinationsThe same control logic repeated again

N × M pipelines repeated the same operating logic

02 / The reusable platform

One reusable layer replaced custom builds

Connections, configuration, execution and operations came together as one reusable product platform.

01SourcesBigQuery · AEP · REST APIs · other sources
02SegmentSync coreConnections · schema discovery · audience builder · mapping · sync projects · scheduler · checkpoint state
03DestinationsOracle Eloqua · AEP · REST APIs

RabbitMQ · PHP workers · retries · run history + logs

03 / Execution architecture

Scheduled syncs ran through one execution layer

The scheduler loaded configuration and checkpoints, then queued concurrent work before destination execution and state updates.

01Source systemsGoogle BigQuery · AEP datasets · REST APIs
02Scheduler + projectLoad sync project · read checkpoint · enqueue jobs
03Queue + concurrent workRabbitMQ · worker pool · read · transform · map
04Destination + stateWrite records · success update · checkpoint · retry state

Run history and monitoring made broker work, retries, checkpoints and recovery visible

04 / Audience builder

Audience rules became a visual workflow

Teams could discover available fields, compose nested rules without writing SQL, save an audience and reuse it in scheduled syncs.

01Choose a source datasetDiscover available BigQuery or AEP fields
02Define rules visuallyFields · operators · nested groups
03Save the audienceCreate a reusable activation object
04Attach to scheduled syncsReuse the same definition in recurring projects

05 / Configurable activation

Mappings made every activation configurable

Each sync project could align different source and destination schemas without hard-coding a new pipeline.

01Source fieldsemail · audience_id · qualified_at
02MappingDirect · rename · date transform
03Destination fieldsEmailAddress · AudienceId · QualificationTimestamp

Configuration, not hard-coded pairs

06 / Reliable execution

Checkpointed execution kept recurring syncs recoverable

The platform read only what changed and advanced state only after successful destination processing.

01Read checkpointResume from known state
02Identify deltaNew or changed records
03Process concurrentlyMultiple PHP workers
04Destination writeMap + write records
05Commit checkpointAdvance only after success

Failure · retry / error state · run history · logs · checkpoint safety

07 / Operational evidence

Every sync exposed its operating health

Execution runs, event logs, project health and checkpoints made successful work, failures and recovery inspectable.

01Execution runsRun ID · status · schedule · duration · records loaded
02Run event logMessage receipt · lock · processing · completion
03Project healthSuccess rate · failures · retries · runtime
04Checkpoint stateThe safe point for the next run

08 / Platform outcome

Integrations became reusable operations

A reusable layer reduced manual movement and made recurring activation operable by design.

01One reusable layerConnections, mappings, schedules and execution patterns reused across projects
02Less manual movementAudiences and datasets moved on recurring schedules
03Operable by designCheckpoints, retries, status, history and logs made syncs recoverable and visible

Configure once. Schedule it. Monitor it. Reuse the integration.