# Bulk Disbursement Integration ### Bulk Payment Design * Provide scalable and fault tolerant design with easy integration * Allow easy plugging with different payment types (Mojaloop, GSMA, Afrimoney and others) * Enable reuse of existing BPMNs to facilitate transfers #### Prioritizing New Use Cases * G2P * P2G #### Bulk Processor Flow ![](/files/I6voe8lHYWVf7MoTNKTo) * Bulk processing is designed as a separate system within Payment Hub EE architecture supporting * File integrity check with MD5/SHA256 checksum when received in channel connector * Bulk processor * Split streamed files and split into sub-batches * Format file contents into a format bulk connectors can parse * Order payments * Publish to Kafka payment topic * Merge back sub-file and sub-batch results into a consolidated response * Bulk connector * Format data based on payment connector requirements * Starting point of initiated payment type-specific workflows * Stream sub-batch transactions from S3 based on metadata from Kafka #### Configurations To provide a granular level of control, the bulk processor supports multiple configurations for different stages like ordering, formatting, splitting etc. * partylookup.enable: used to enable or disable the partylookup stage * approval.enable: used to enable or disable the approval stage * ordering.enable: used to enable or disable the ordering of transactions in a data set based on a field provided. * ordering.field: the field based on which ordering will be done. * formatting.enable: enable or disable the formatting of data set. If formatting is disabled the default formatting standard is used. * formatting.standard: the formatting standard to be used for transactions, possible values are "DEFAULT", "GSMA". * splitting.enable: enable or disable the splitting of the original csv file into sub-batches. Splitting is done on the sub-batch-size field configuration. * splitting.sub-batch-size: the size of the single sub batch. * mergeback.enable: enable or disable the merging back of the sub batches result. * success-threshold-check.enable: enable or disable the success threshold check, use this configuration to make sure at least x percentage of transaction is successful in a batch. * success-threshold-check.success-rate: a percentage value that will be used while making the threshold check. * acknowledgement.enable: enable or disable the acknowledgement part of the workflow. **Key Considerations:** * Kafka provides auto partitioning within topics based on throughput and data * Scalable and fault-tolerant system provided by the multi-cluster setup * Handle backpressure from Zeebe and throttle workflow initiation * Kafka will act as a buffer in this case * Advantageous when Zeebe is overloaded * Due to resource limitations * Or not fast enough scaling * Reduces cascading Denial of Service for other payment connectors. * Provide a visual workflow representation with Zeebe * Stores every state for workflow instances * Timers surviving node failures without interruptions * Using an efficient, high-performance binary protocol (gRPC) for the clients to connect and receive tasks for execution * Retry mechanism on the entire subprocess * But retry is already in place within subprocesses * Unified payment architecture * Easy integration with AMS if payment staging requires it * In case of beneficiary creation during transfer * Using Raft for log replication * Kafka to write ahead log replication as well. * Easy integration of pre and post-transaction workers * Such as bulk notification #### References: ## API Specification * Provide developer friendly consumer APIs * Enable easy integration by abstracting underlying payment type API complexities * Group common fields across various payment APIs and make em mandatory field * All fields in the spec below are mandatory fields * Allow addition of payment type specific fields as optional ### 1. Bulk Transfer Use below csv file for refernece. ```csv id,request_id,payment_mode,payer_identifier_type,payer_identifier,payee_identifier_type,payee_identifier,amount,currency,note,program_shortcode,cycle 0,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,slcb,accountNumber,003001003874120160,accountNumber,927549027483,850,USD,Test Payee Payment,MDM,1 1,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,slcb,accountNumber,003001003873110196,accountNumber,927549027483,222,USD,Test Payee Payment,MDM,1 2,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,slcb,accountNumber,003001003874120160,accountNumber,927549027483,850,USD,Test Payee Payment,MDM,1 3,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,slcb,accountNumber,003001003873110196,accountNumber,927549027483,222,USD,Test Payee Payment,MDM,1 4,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,mojaloop,accountNumber,003001003874120160,mosip,437892675228,850,USD,Test Payee Payment,COV,1 5,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,mojaloop,accountNumber,003001003873110196,mosip,437892675228,222,USD,Test Payee Payment,COV,1 6,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,mojaloop,accountNumber,003001003874120160,mosip,437892675228,850,USD,Test Payee Payment,COV,1 7,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,mojaloop,accountNumber,003001003873110196,mosip,437892675228,222,USD,Test Payee Payment,COV,1 8,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,gsma,accountNumber,003001003874120160,msisdn,8837461856,850,USD,Test Payee Payment,CWP,2 9,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,gsma,accountNumber,003001003873110196,msisdn,8837461856,222,USD,Test Payee Payment,CWP,2 10,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,gsma,accountNumber,003001003874120160,msisdn,8837461856,850,USD,Test Payee Payment,CWP,2 11,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,gsma,accountNumber,003001003873110196,msisdn,8837461856,222,USD,Test Payee Payment,CWP,2 ``` #### Use the below request along with above CSV file {% code overflow="wrap" %} ```bash curl --location --request POST 'https://bulk-connector.sandbox.fynarfin.io/batchtransactions?type=csv' \ --header 'Platform-TenantId: ibank-usa' \ --header 'X-CorrelationID: 3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8' \ --header 'filename: ph-ee-bulk-demo-6.csv --form 'data=@"ph-ee-bulk-demo-6.csv"' ``` {% endcode %} #### Use the below request to start a batch only using the filename {% code overflow="wrap" %} ```bash curl --location --request POST 'https://bulk-connector.sandbox.fynarfin.io/batchtransactions?type=raw' \ --header 'Platform-TenantId: ibank-usa' \ --header 'X-CorrelationID: 3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8' \ --header 'filename: ph-ee-bulk-demo-6.csv' ``` {% endcode %} #### Use the below request to pass transaction data in the body of the request ```bash curl --location --request POST 'https://bulk-connector.sandbox.fynarfin.io/batchtransactions?type=raw' \ --header 'Platform-TenantId: ibank-usa' \ --header 'X-CorrelationID: 3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8' \ --header 'Content-Type: application/json' \ --data-raw '[ { "creditParty": [ { "key": "msisdn", "value": "8837461856" } ], "debitParty": [ { "key": "accountnumber", "value": "003001003874120160" } ], "subType": "SLCB", "amount": "820.00", "currency": "RWF", "descriptionText": "Test Payment" } ]' ``` ```json { "batch_id": "738bd830-3bd9-4511-bb42-3d0f5798e014", "request_id": "3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8", "status": "queued" } ``` ### 2. Bulk Transfer Status {% code overflow="wrap" %} ```bash curl --location --request GET 'https://ops-bk.sandbox.fynarfin.io/api/v1/batchtransactions/3112a0ba-0733-4133-ae24-fc3310cb7dfe' \ --header 'Platform-TenantId: ibank-usa' \ --header 'Authorization: Bearer token' ``` {% endcode %} 
{  
  "batch_id": "UUID",
  "request_id": "UUID",
  "notes": "Bulk transfers",
  "status": "processing",
  "mode": "{payment_mode}",
  "purpose": "refund",
  "total": 1000,
  "successful": 980,
  "failed": 20,
  "file": "S3 link", 
  "created_at": 1545383037,
  "totalAmount": "1050000.00",
  "successfulAmount": "975000.00",
  "failedAmount": "25000.00",
  "pendingAmount": "50000.00",
  "currency": "SLE"
}
#### Response File ```csv id,request_id,payment_mode,payer_identifier_type,payer_identifier,payee_identifier_type,payee_identifier,amount,currency,note,program_shortcode,cycle,status,error_code,error_description 0,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,slcb,accountNumber,003001003874120160,accountNumber,927549027483,850,USD,Test Payee Payment,MDM,1,SUCCESS,, 1,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,slcb,accountNumber,003001003873110196,accountNumber,927549027483,222,USD,Test Payee Payment,MDM,1,SUCCESS,, 2,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,slcb,accountNumber,003001003874120160,accountNumber,927549027483,850,USD,Test Payee Payment,MDM,1,SUCCESS,, 3,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,slcb,accountNumber,003001003873110196,accountNumber,927549027483,222,USD,Test Payee Payment,MDM,1,FAILURE,422,Inactive Account 4,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,mojaloop,accountNumber,003001003874120160,mosip,437892675228,850,USD,Test Payee Payment,COV,1,SUCCESS,, 5,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,mojaloop,accountNumber,003001003873110196,mosip,437892675228,222,USD,Test Payee Payment,COV,1,SUCCESS,, 6,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,mojaloop,accountNumber,003001003874120160,mosip,437892675228,850,USD,Test Payee Payment,COV,1,SUCCESS,, 7,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,mojaloop,accountNumber,003001003873110196,mosip,437892675228,222,USD,Test Payee Payment,COV,1,PENDING,, 8,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,gsma,accountNumber,003001003874120160,msisdn,8837461856,850,USD,Test Payee Payment,CWP,2,SUCCESS,, 9,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,gsma,accountNumber,003001003873110196,msisdn,8837461856,222,USD,Test Payee Payment,CWP,2,SUCCESS,, 10,f1e22fe3-9740-4fba-97b6-78f43bfa7f2f,gsma,accountNumber,003001003874120160,msisdn,8837461856,850,USD,Test Payee Payment,CWP,2,PENDING,, 11,72aa3ea4-e6f6-4880-877f-39f6ac4d052e,gsma,accountNumber,003001003873110196,msisdn,8837461856,222,USD,Test Payee Payment,CWP,2,SUCCESS,, ``` ### 3. Batch Details {% code overflow="wrap" %} ```bash curl --location --request GET 'https://ops-bk.sandbox.fynarfin.io/api/v1/batchtransactions/45e2baca-b087-4d90-8392-da2961f9b9ed/detail?pageNo=1&pageSize=10' \ --header 'Platform-TenantId: ibank-usa' \ --header 'Authorization: Bearer Token' ``` {% endcode %} ```json [ { "id": 568, "workflowInstanceKey": 2251799907439003, "transactionId": "e5eea064-1445-4d32-bc55-bd9826c779a0", "startedAt": 1629130966000, "completedAt": 1629130967000, "status": "IN_PROGRESS", "statusDetail": null, "payeeDfspId": "giraffe-bank", "payeePartyId": "7854278651", "payeePartyIdType": "MSISDN", "payeeFee": null, "payeeFeeCurrency": null, "payeeQuoteCode": null, "payerDfspId": "gorilla-bank", "payerPartyId": "7543010", "payerPartyIdType": "MSISDN", "payerFee": null, "payerFeeCurrency": null, "payerQuoteCode": null, "amount": 448, "currency": "USD", "direction": "OUTGOING", "errorInformation": "{\\\"errorInformation\\\":{\\\"errorDescription\\\":\\\"Invalid responseCode 500 for transfer on PAYER side, transactionId: e5eea064-1445-4d32-bc55-bd9826c779a0 Message: {\\\\\\\"timestamp\\\\\\\":1629130966891,\\\\\\\"status\\\\\\\":500,\\\\\\\"error\\\\\\\":\\\\\\\"Internal Server Error\\\\\\\",\\\\\\\"message\\\\\\\":\\\\\\\"\\\\\\\",\\\\\\\"path\\\\\\\":\\\\\\\"/fineract-provider/api/v1/interoperation/transfers\\\\\\\"}\\\",\\\"errorCode\\\":\\\"4101\\\"}}", "batchId": "c02a14f0-5e7e-44a1-88eb-5584a21e6f28" } ] ``` ### 4. Get Batches {% code overflow="wrap" %} ```bash curl --location --request GET 'http://ops-bk.sandbox.fynarfin.io/api/v1/batches?page=2&size=10&sortedBy=requestFile&sortedOrder=asc' \ --header 'Platform-TenantId: ibank-usa' ``` {% endcode %} {% code overflow="wrap" %} ```json { "content": [ { "id": 57, "batchId": "6bdd4e23-9357-481b-b814-b678b146a01b", "subBatchId": null, "requestId": "3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8", "requestFile": "1661791795354_ph-ee-bulk-demo-6.csv", "totalTransactions": 11, "total": 12, "ongoing": 0, "failed": 3, "successful": 9, "totalAmount": 6432, "successfulAmount": 4510, "pendingAmount": 0, "failedAmount": 1922, "result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/1663238730228_6bdd4e23-9357-481b-b814-b678b146a01b.csv", "resultGeneratedAt": null, "note": null, "workflowKey": null, "workflowInstanceKey": 2251799813689497, "startedAt": 1661791796000, "completedAt": 1661791796000 }, { "id": 29, "batchId": "837f0b1a-d06c-4a7b-a6bb-a274e7e3cf43", "subBatchId": null, "requestId": "1aff3859d4e8-fab3a4d5-0f4f-4e78-b6b5", "requestFile": "1661793332698_ph-ee-bulk-demo-6.csv", "totalTransactions": 12, "ongoing": 0, "failed": 12, "completed": 0, "totalAmount": 3810, "ongoingAmount": null, "failedAmount": 3810, "completedAmount": null, "result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/5618434551624_39484844071623_response.csv", "resultGeneratedAt": 1661844072000, "note": null, "workflowKey": null, "workflowInstanceKey": 2251799813692279, "startedAt": 1661793333000, "completedAt": 1661793333000 }, { "id": 33, "batchId": "de2b9caf-07f0-43d0-ab98-34e070f52913", "subBatchId": null, "requestId": "1", "requestFile": "1661798671312_1", "totalTransactions": 50, "ongoing": null, "failed": null, "completed": 50, "totalAmount": 12490, "ongoingAmount": null, "failedAmount": null, "completedAmount": 12490, "result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/1661873857624_1661843746423_response.csv", "resultGeneratedAt": null, "note": null, "workflowKey": null, "workflowInstanceKey": 2251799813699494, "startedAt": 1661798674000, "completedAt": null }, { "id": 34, "batchId": "eb94687b-5eee-4bc9-ad8a-e89bf0604dc1", "subBatchId": null, "requestId": "3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8", "requestFile": "1661840850289_ph-ee-bulk-demo-6.csv", "totalTransactions": null, "ongoing": null, "failed": null, "completed": null, "totalAmount": null, "ongoingAmount": null, "failedAmount": null, "completedAmount": null, "result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/1661849863624_1661983851623_response.csv", "resultGeneratedAt": null, "note": null, "workflowKey": null, "workflowInstanceKey": 2251799813750207, "startedAt": 1661840852000, "completedAt": 1661840853000 }, { "id": 38, "batchId": "7f2a7366-3cc1-4293-8cc7-21f862872f7b", "subBatchId": null, "requestId": "3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8", "requestFile": "1661840967146_ph-ee-bulk-demo-6.csv", "totalTransactions": null, "ongoing": null, "failed": null, "completed": null, "totalAmount": null, "ongoingAmount": null, "failedAmount": null, "completedAmount": null, "result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/1661883987873_1661844071623_response.csv", "resultGeneratedAt": null, "note": null, "workflowKey": null, "workflowInstanceKey": 2251799813750554, "startedAt": 1661840967000, "completedAt": 1661840968000 }, { "id": 42, "batchId": "a7159479-e0d8-4a77-8f92-8b903b3173b7", "subBatchId": null, "requestId": "3a4dfab5-0f4f-4e78-b6b5-1aff3859d4e8", "requestFile": "1661843665396_ph-ee-bulk-demo-6.csv", "totalTransactions": 12, "ongoing": 0, "failed": 12, "completed": 0, "totalAmount": null, "ongoingAmount": null, "failedAmount": null, "completedAmount": null, "result_file": "https://paymenthub-ee-dev.s3.us-east-2.amazonaws.com/1661844087873_1661844087873_response.csv", "resultGeneratedAt": 1661844088000, "note": null, "workflowKey": null, "workflowInstanceKey": 2251799813754434, "startedAt": 1661843667000, "completedAt": 1661843668000 } ], "totalElements": 79, "totalPages": 8, "last": false, "numberOfElements": 10, "sort": [ { "direction": "ASC", "property": "requestFile", "ignoreCase": false, "nullHandling": "NATIVE", "descending": false, "ascending": true } ], "first": false, "size": 10, "number": 2 } ``` {% endcode %} ### 5. Download Template {% code overflow="wrap" %} ```bash curl --location --request GET 'https://bulk-connector.sandbox.fynarfin.io/template?types=PROGRAM,MOJALOOP' ``` {% endcode %} {% code title="Template.csv" %} ```csv id,request_id,payment_mode,payer_identifier_type,payer_identifier,payee_identifier_type,payee_identifier,amount,currency,note,program_shortcode,cycle ``` {% endcode %} ## Payment Routing * Route enables user to split payments received and transfer the funds to various vendors/beneficiaries. * Sample request body ```json { "amount": 2000, "currency": "INR", "transfers": [ { "account": "acc_CPRsN1LkFccllA", "amount": 1000, "currency": "INR", "notes": { "branch": "Acme Corp Bangalore North", "name": "Gaurav Kumar" }, }, { "account": "acc_CNo3jSI8OkFJJJ", "amount": 1000, "currency": "INR", "notes": { "branch": "Acme Corp Bangalore South", "name": "Saurav Kumar" }, } ] } ``` * With Auto Payout * Targeting individuals * Money is transferred directly to bank accounts / mobile money wallets from the sender float account. * No involvement of AMS. * Without Auto Payout * Targeting organisations * Money is transferred to Fineract wallets of the receivers. * Receiver can choose if they want to withdraw the money out of the wallet * Or consider using the money in float/current/checking account for L2 beneficiaries (clear invoices, payroll) seamlessly in one platform (L1 beneficiary amount remains in closed loop) * This provides an advantage to beneficiary orgs to manage their payouts if they have a legacy current/checking account with mostly manual payout processing. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://mifos.gitbook.io/docs/payment-hub-ee/overview/bulk-disbursement-integration.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.