Orderful Status - Trading Partner Delivery Services

Trading Partner Delivery Services - Partial Outage

Incident Report for Orderful

Postmortem

Summary: Our VAN provider changed their process to handle duplicate transactions on January 13, 2020. After the change, several customers connected with their trading partners through our VAN provider were not sent inbound transactions. The investigation has shown that the change done by our VAN provider changed the behavior of the polling system that was configured. This resulted in some inbound transactions being removed from the mailbox before being processed by the polling system.

Permanent solution: We made a behavior change of the polling system of our VAN provider mailbox so that inbound transactions stay in the mailbox until they are processed by the polling system. We see that the issue has been resolved as Inbound transactions are coming through as expected, but we’ll monitor the processing of the VAN mailbox over the next days

Posted Jan 16, 2020 - 19:34 UTC

Resolved

This incident has been resolved and all affected transactions has been reprocessed to Orderful. The monitoring job is complete and Inbound files are successfully flowing in from the VAN connections.

We will provide the details of the issue and resolution momentarily.

Posted Jan 16, 2020 - 04:40 UTC

Monitoring

We have identified the root cause and have deployed the fix. We are seeing Inbound transactions being successfully downloaded from VAN connections to Orderful.

For failed transactions, we are currently working closely with our VAN provider to reprocess all affected transactions.

Our team will continue to monitor the transactions and alerts to ensure that issue is resolved.

Posted Jan 15, 2020 - 23:41 UTC

Update

We are still investigating this issue. What we know: Our VAN provider made a change to their API to transfer files to us. They made this change in an attempt to fix other issues; however, it resulted in an outage that has prevented inbound files from being transferred.

We are working to find the root cause and are attempting to determine the status of the missing files. If this fix takes some time, we will be looking have the files resent manually by our VAN provider.

Posted Jan 15, 2020 - 21:59 UTC

Investigating

We’re currently experiencing degraded performance issues with our VAN Delivery Service. Our team is currently working to restore normal performance levels. We apologize for any inconvenience.

Customers with Trading Partners using VAN connectivity are impacted. All inbound parcels are being queued and will be processed as soon as service is restored. Next update in 30 mins - 1 hour.

Posted Jan 15, 2020 - 20:30 UTC

This incident affected: Transaction Delivery (Trading Partner Delivery Services).