Comment on page
Bulk Disbursement Integration
- Provide scalable and fault tolerant design with easy integration
- Allow easy plugging with different payment types (Mojaloop, GSMA, Afrimoney and others)
- Enable reuse of existing BPMNs to facilitate transfers
- G2P
- P2G
- Bulk processing is designed as a separate system within Payment Hub EE architecture supporting
- File integrity check with MD5/SHA256 checksum when received in channel connector
- Bulk processor
- Split streamed files and split into sub-batches
- Format file contents into a format bulk connectors can parse
- Order payments
- Publish to Kafka payment topic
- Merge back sub-file and sub-batch results into a consolidated response
- Bulk connector
- Format data based on payment connector requirements
- Starting point of initiated payment type-specific workflows
- Stream sub-batch transactions from S3 based on metadata from Kafka
To provide a granular level of control, the bulk processor supports multiple configurations for different stages like ordering, formatting, splitting etc.
- partylookup.enable: used to enable or disable the partylookup stage
- approval.enable: used to enable or disable the approval stage
- ordering.enable: used to enable or disable the ordering of transactions in a data set based on a field provided.
- ordering.field: the field based on which ordering will be done.
- formatting.enable: enable or disable the formatting of data set. If formatting is disabled the default formatting standard is used.
- formatting.standard: the formatting standard to be used for transactions, possible values are "DEFAULT", "GSMA".
- splitting.enable: enable or disable the splitting of the original csv file into sub-batches. Splitting is done on the sub-batch-size field configuration.
- splitting.sub-batch-size: the size of the single sub batch.
- mergeback.enable: enable or disable the merging back of the sub batches result.
- success-threshold-check.enable: enable or disable the success threshold check, use this configuration to make sure at least x percentage of transaction is successful in a batch.
- success-threshold-check.success-rate: a percentage value that will be used while making the threshold check.
- acknowledgement.enable: enable or disable the acknowledgement part of the workflow.
Key Considerations:
- Kafka provides auto partitioning within topics based on throughput and data
- Scalable and fault-tolerant system provided by the multi-cluster setup
- Handle backpressure from Zeebe and throttle workflow initiation
- Kafka will act as a buffer in this case
- Advantageous when Zeebe is overloaded
- Due to resource limitations
- Or not fast enough scaling
- Reduces cascading Denial of Service for other payment connectors.
- Provide a visual workflow representation with Zeebe
- Stores every state for workflow instances
- Timers surviving node failures without interruptions
- Using an efficient, high-performance binary protocol (gRPC) for the clients to connect and receive tasks for execution
- Retry mechanism on the entire subprocess
- But retry is already in place within subprocesses
- Unified payment architecture
- Easy integration with AMS if payment staging requires it
- In case of beneficiary creation during transfer
- Using Raft for log replication
- Kafka to write ahead log replication as well.
- Easy integration of pre and post-transaction workers
- Such as bulk notification
- Provide developer friendly consumer APIs
- Enable easy integration by abstracting underlying payment type API complexities
- Group common fields across various payment APIs and make em mandatory field
- All fields in the spec below are mandatory fields
- Allow addition of payment type specific fields as optional
Use below csv file for refernece.
id,request_id,payment_mode,payer_identifier_type,payer_identifier,payee_identifier_type,payee_identifier,amount,currency,note,program_shortcode,cycle
0,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,slcb,accountNumber,003001003874120160,accountNumber,927549027483,850,USD,Test Payee Payment,MDM,1
1,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,slcb,accountNumber,003001003873110196,accountNumber,927549027483,222,USD,Test Payee Payment,MDM,1
2,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,slcb,accountNumber,003001003874120160,accountNumber,927549027483,850,USD,Test Payee Payment,MDM,1
3,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,slcb,accountNumber,003001003873110196,accountNumber,927549027483,222,USD,Test Payee Payment,MDM,1
4,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,mojaloop,accountNumber,003001003874120160,mosip,437892675228,850,USD,Test Payee Payment,COV,1
5,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,mojaloop,accountNumber,003001003873110196,mosip,437892675228,222,USD,Test Payee Payment,COV,1
6,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,mojaloop,accountNumber,003001003874120160,mosip,437892675228,850,USD,Test Payee Payment,COV,1
7,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,mojaloop,accountNumber,003001003873110196,mosip,437892675228,222,USD,Test Payee Payment,COV,1
8,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,gsma,accountNumber,003001003874120160,msisdn,8837461856,850,USD,Test Payee Payment,CWP,2
9,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,gsma,accountNumber,003001003873110196,msisdn,8837461856,222,USD,Test Payee Payment,CWP,2
10,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,gsma,accountNumber,003001003874120160,msisdn,8837461856,850,USD,Test Payee Payment,CWP,2
11,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,gsma,accountNumber,003001003873110196,msisdn,8837461856,222,USD,Test Payee Payment,CWP,2
curl --location --request POST 'https://bulk-connector.sandbox.fynarfin.io/batchtransactions?type=csv' \
--header 'Platform-TenantId: ibank-usa' \
--header 'X-CorrelationID: 3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8' \
--header 'filename: ph-ee-bulk-demo-6.csv
--form 'data=@"ph-ee-bulk-demo-6.csv"'
curl --location --request POST 'https://bulk-connector.sandbox.fynarfin.io/batchtransactions?type=raw' \
--header 'Platform-TenantId: ibank-usa' \
--header 'X-CorrelationID: 3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8' \
--header 'filename: ph-ee-bulk-demo-6.csv'
curl --location --request POST 'https://bulk-connector.sandbox.fynarfin.io/batchtransactions?type=raw' \
--header 'Platform-TenantId: ibank-usa' \
--header 'X-CorrelationID: 3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8' \
--header 'Content-Type: application/json' \
--data-raw '[
{
"creditParty": [
{
"key": "msisdn",
"value": "8837461856"
}
],
"debitParty": [
{
"key": "accountnumber",
"value": "003001003874120160"
}
],
"subType": "SLCB",
"amount": "820.00",
"currency": "RWF",
"descriptionText": "Test Payment"
}
]'
{
"batch_id": "738bd830-3bd9-4511-bb42-3d0f5798e014",
"request_id": "3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8",
"status": "queued"
}
curl --location --request GET 'https://ops-bk.sandbox.fynarfin.io/api/v1/batchtransactions/3112a0ba-0733-4133-ae24-fc3310cb7dfe' \
--header 'Platform-TenantId: ibank-usa' \
--header 'Authorization: Bearer token'
{
"batch_id": "UUID",
"request_id": "UUID",
"notes": "Bulk transfers",
"status": "processing",
"mode": "{payment_mode}",
"purpose": "refund",
"total": 1000,
"successful": 980,
"failed": 20,
"file": "S3 link",
"created_at": 1545383037,
"totalAmount": "1050000.00",
"successfulAmount": "975000.00",
"failedAmount": "25000.00",
"pendingAmount": "50000.00",
"currency": "SLE"
}
id,request_id,payment_mode,payer_identifier_type,payer_identifier,payee_identifier_type,payee_identifier,amount,currency,note,program_shortcode,cycle,status,error_code,error_description
0,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,slcb,accountNumber,003001003874120160,accountNumber,927549027483,850,USD,Test Payee Payment,MDM,1,SUCCESS,,
1,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,slcb,accountNumber,003001003873110196,accountNumber,927549027483,222,USD,Test Payee Payment,MDM,1,SUCCESS,,
2,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,slcb,accountNumber,003001003874120160,accountNumber,927549027483,850,USD,Test Payee Payment,MDM,1,SUCCESS,,
3,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,slcb,accountNumber,003001003873110196,accountNumber,927549027483,222,USD,Test Payee Payment,MDM,1,FAILURE,422,Inactive Account
4,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,mojaloop,accountNumber,003001003874120160,mosip,437892675228,850,USD,Test Payee Payment,COV,1,SUCCESS,,
5,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,mojaloop,accountNumber,003001003873110196,mosip,437892675228,222,USD,Test Payee Payment,COV,1,SUCCESS,,
6,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,mojaloop,accountNumber,003001003874120160,mosip,437892675228,850,USD,Test Payee Payment,COV,1,SUCCESS,,
7,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,mojaloop,accountNumber,003001003873110196,mosip,437892675228,222,USD,Test Payee Payment,COV,1,PENDING,,
8,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,gsma,accountNumber,003001003874120160,msisdn,8837461856,850,USD,Test Payee Payment,CWP,2,SUCCESS,,
9,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,gsma,accountNumber,003001003873110196,msisdn,8837461856,222,USD,Test Payee Payment,CWP,2,SUCCESS,,
10,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,gsma,accountNumber,003001003874120160,msisdn,8837461856,850,USD,Test Payee Payment,CWP,2,PENDING,,
11,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,gsma,accountNumber,003001003873110196,msisdn,8837461856,222,USD,Test Payee Payment,CWP,2,SUCCESS,,
curl --location --request GET 'https://ops-bk.sandbox.fynarfin.io/api/v1/batchtransactions/45e2baca-b087-4d90-8392-da2961f9b9ed/detail?pageNo=1&pageSize=10' \
--header 'Platform-TenantId: ibank-usa' \
--header 'Authorization: Bearer Token'
[
{
"id": 568,
"workflowInstanceKey": 2251799907439003,
"transactionId": "e5eea064-1445-4d32-bc55-bd9826c779a0",
"startedAt": 1629130966000,
"completedAt": 1629130967000,
"status": "IN_PROGRESS",
"statusDetail": null,
"payeeDfspId": "giraffe-bank",
"payeePartyId": "7854278651",
"payeePartyIdType": "MSISDN",
"payeeFee": null,
"payeeFeeCurrency": null,
"payeeQuoteCode": null,
"payerDfspId": "gorilla-bank",
"payerPartyId": "7543010",
"payerPartyIdType": "MSISDN",
"payerFee": null,
"payerFeeCurrency": null,
"payerQuoteCode": null,
"amount": 448,
"currency": "USD",
"direction": "OUTGOING",
"errorInformation": "{\\\"errorInformation\\\":{\\\"errorDescription\\\":\\\"Invalid responseCode 500 for transfer on PAYER side, transactionId: e5eea064-1445-4d32-bc55-bd9826c779a0 Message: {\\\\\\\"timestamp\\\\\\\":1629130966891,\\\\\\\"status\\\\\\\":500,\\\\\\\"error\\\\\\\":\\\\\\\"Internal Server Error\\\\\\\",\\\\\\\"message\\\\\\\":\\\\\\\"\\\\\\\",\\\\\\\"path\\\\\\\":\\\\\\\"/fineract-provider/api/v1/interoperation/transfers\\\\\\\"}\\\",\\\"errorCode\\\":\\\"4101\\\"}}",
"batchId": "c02a14f0-5e7e-44a1-88eb-5584a21e6f28"
}
]
curl --location --request GET 'http://ops-bk.sandbox.fynarfin.io/api/v1/batches?page=2&size=10&sortedBy=requestFile&sortedOrder=asc' \
--header 'Platform-TenantId: ibank-usa'
{
"content": [
{
"id": 57,
"batchId": "6bdd4e23-9357-481b-b814-b678b146a01b",
"subBatchId": null,
"requestId": "3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8",
"requestFile": "1661791795354_ph-ee-bulk-demo-6.csv",
"totalTransactions": 11,
"total": 12,
"ongoing": 0,
"failed": 3,
"successful": 9,
"totalAmount": 6432,
"successfulAmount": 4510,
"pendingAmount": 0,
"failedAmount": 1922,
"result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/1663238730228_6bdd4e23-9357-481b-b814-b678b146a01b.csv",
"resultGeneratedAt": null,
"note": null,
"workflowKey": null,
"workflowInstanceKey": 2251799813689497,
"startedAt": 1661791796000,
"completedAt": 1661791796000
},
{
"id": 29,
"batchId": "837f0b1a-d06c-4a7b-a6bb-a274e7e3cf43",
"subBatchId": null,
"requestId": "1aff3859d4e8-fab3a4d5-0f4f-4e78-b6b5",
"requestFile": "1661793332698_ph-ee-bulk-demo-6.csv",
"totalTransactions": 12,
"ongoing": 0,
"failed": 12,
"completed": 0,
"totalAmount": 3810,
"ongoingAmount": null,
"failedAmount": 3810,
"completedAmount": null,
"result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/5618434551624_39484844071623_response.csv",
"resultGeneratedAt": 1661844072000,
"note": null,
"workflowKey": null,
"workflowInstanceKey": 2251799813692279,
"startedAt": 1661793333000,
"completedAt": 1661793333000
},
{
"id": 33,
"batchId": "de2b9caf-07f0-43d0-ab98-34e070f52913",
"subBatchId": null,
"requestId": "1",
"requestFile": "1661798671312_1",
"totalTransactions": 50,
"ongoing": null,
"failed": null,
"completed": 50,
"totalAmount": 12490,
"ongoingAmount": null,
"failedAmount": null,
"completedAmount": 12490,
"result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/1661873857624_1661843746423_response.csv",
"resultGeneratedAt": null,
"note": null,
"workflowKey": null,
"workflowInstanceKey": 2251799813699494,
"startedAt": 1661798674000,
"completedAt": null
},
{
"id": 34,
"batchId": "eb94687b-5eee-4bc9-ad8a-e89bf0604dc1",
"subBatchId": null,
"requestId": "3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8",
"requestFile": "1661840850289_ph-ee-bulk-demo-6.csv",
"totalTransactions": null,
"ongoing": null,
"failed": null,
"completed": null,
"totalAmount": null,
"ongoingAmount": null,
"failedAmount": null,
"completedAmount": null,
"result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/1661849863624_1661983851623_response.csv",
"resultGeneratedAt": null,
"note": null,
"workflowKey": null,
"workflowInstanceKey": 2251799813750207,
"startedAt": 1661840852000,
"completedAt": 1661840853000
},
{
"id": 38,
"batchId": "7f2a7366-3cc1-4293-8cc7-21f862872f7b",
"subBatchId": null,
"requestId": "3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8",
"requestFile": "1661840967146_ph-ee-bulk-demo-6.csv",
"totalTransactions": null,
"ongoing": null,
"failed": null,
"completed": null,
"totalAmount": null,
"ongoingAmount": null,
"failedAmount": null,
"completedAmount": null,
"result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/1661883987873_1661844071623_response.csv",
"resultGeneratedAt": null,
"note": null,
"workflowKey": null,
"workflowInstanceKey": 2251799813750554,
"startedAt": 1661840967000,
"completedAt": 1661840968000
},
{
"id": 42,
"batchId": "a7159479-e0d8-4a77-8f92-8b903b3173b7",
"subBatchId": null,
"requestId": "3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8",
"requestFile": "1661843665396_ph-ee-bulk-demo-6.csv",
"totalTransactions": 12,
"ongoing": 0,
"failed": 12,
"completed": 0,
"totalAmount": null,
"ongoingAmount": null,
"failedAmount": null,
"completedAmount": null,
"result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/1661844087873_1661844087873_response.csv",
"resultGeneratedAt": 1661844088000,
"note": null,
"workflowKey": null,
"workflowInstanceKey": 2251799813754434,
"startedAt": 1661843667000,
"completedAt": 1661843668000
}
],
"totalElements": 79,
"totalPages": 8,
"last": false,
"numberOfElements": 10,
"sort": [
{
"direction": "ASC",
"property": "requestFile",
"ignoreCase": false,
"nullHandling": "NATIVE",
"descending": false,
"ascending": true
}
],
"first": false,
"size": 10,
"number": 2
}
curl --location --request GET 'https://bulk-connector.sandbox.fynarfin.io/template?types=PROGRAM,MOJALOOP'
Template.csv
id,request_id,payment_mode,payer_identifier_type,payer_identifier,payee_identifier_type,payee_identifier,amount,currency,note,program_shortcode,cycle
- Route enables user to split payments received and transfer the funds to various vendors/beneficiaries.
- Sample request body
{
"amount": 2000,
"currency": "INR",
"transfers": [
{
"account": "acc_CPRsN1LkFccllA",
"amount": 1000,
"currency": "INR",
"notes": {
"branch": "Acme Corp Bangalore North",
"name": "Gaurav Kumar"
},
},
{
"account": "acc_CNo3jSI8OkFJJJ",
"amount": 1000,
"currency": "INR",
"notes": {
"branch": "Acme Corp Bangalore South",
"name": "Saurav Kumar"
},
}
]
}
- With Auto Payout
- Targeting individuals
- Money is transferred directly to bank accounts / mobile money wallets from the sender float account.
- No involvement of AMS.
- Without Auto Payout
- Targeting organisations
- Money is transferred to Fineract wallets of the receivers.
- Receiver can choose if they want to withdraw the money out of the wallet
- Or consider using the money in float/current/checking account for L2 beneficiaries (clear invoices, payroll) seamlessly in one platform (L1 beneficiary amount remains in closed loop)
- This provides an advantage to beneficiary orgs to manage their payouts if they have a legacy current/checking account with mostly manual payout processing.
Last modified 15d ago