Aspose Words API error

nikhil.goyal · February 25, 2025, 11:52am

We are facing api error unexpectedly from aspose API. We are using aspose_words_cloud gem version 22.9.
We are getting this error

Exception
Faraday::ConnectionFailed
Error
end of file reached
/app/vendor/bundle/ruby/3.1.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:237:in `rbuf_fill'
/app/vendor/bundle/ruby/3.1.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:199:in `readuntil'
/app/vendor/bundle/ruby/3.1.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:209:in `readline'
/app/vendor/ruby-3.1.2/lib/ruby/3.1.0/net/http/response.rb:322:in `read_chunked'
/app/vendor/ruby-3.1.2/lib/ruby/3.1.0/net/http/response.rb:292:in `block in read_body_0'
/app/vendor/ruby-3.1.2/lib/ruby/3.1.0/net/http/response.rb:285:in `inflater'
/app/vendor/ruby-3.1.2/lib/ruby/3.1.0/net/http/response.rb:290:in `read_body_0'
/app/vendor/ruby-3.1.2/lib/ruby/3.1.0/net/http/response.rb:210:in `read_body'
/app/vendor/ruby-3.1.2/lib/ruby/3.1.0/net/http/response.rb:235:in `body'
/app/vendor/ruby-3.1.2/lib/ruby/3.1.0/net/http/response.rb:168:in `reading_body'
/app/vendor/ruby-3.1.2/lib/ruby/3.1.0/net/http.rb:1583:in `transport_request'
/app/vendor/ruby-3.1.2/lib/ruby/3.1.0/net/http.rb:1539:in `request'
/app/vendor/bundle/ruby/3.1.0/gems/scout_apm-5.3.5/lib/scout_apm/instruments/net_http.rb:34:in `block in request_with_scout_instruments'
/app/vendor/bundle/ruby/3.1.0/gems/scout_apm-5.3.5/lib/scout_apm/tracer.rb:34:in `instrument'
/app/vendor/bundle/ruby/3.1.0/gems/scout_apm-5.3.5/lib/scout_apm/tracer.rb:44:in `instrument'
/app/vendor/bundle/ruby/3.1.0/gems/scout_apm-5.3.5/lib/scout_apm/instruments/net_http.rb:33:in `request_with_scout_instruments'
/app/vendor/bundle/ruby/3.1.0/gems/sentry-ruby-5.5.0/lib/sentry/net/http.rb:34:in `request'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-net_http-1.0.1/lib/faraday/adapter/net_http.rb:152:in `block in request_via_request_method'
/app/vendor/ruby-3.1.2/lib/ruby/3.1.0/net/http.rb:966:in `start'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-net_http-1.0.1/lib/faraday/adapter/net_http.rb:146:in `request_via_request_method'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-net_http-1.0.1/lib/faraday/adapter/net_http.rb:131:in `request_with_wrapped_block'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-net_http-1.0.1/lib/faraday/adapter/net_http.rb:122:in `perform_request'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-net_http-1.0.1/lib/faraday/adapter/net_http.rb:66:in `block in call'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-1.10.3/lib/faraday/adapter.rb:50:in `connection'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-net_http-1.0.1/lib/faraday/adapter/net_http.rb:64:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-1.10.3/lib/faraday/request/url_encoded.rb:25:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-multipart-1.0.4/lib/faraday/multipart/middleware.rb:28:in `call'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-1.10.3/lib/faraday/rack_builder.rb:154:in `build_response'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-1.10.3/lib/faraday/connection.rb:516:in `run_request'
/app/vendor/bundle/ruby/3.1.0/gems/faraday-1.10.3/lib/faraday/connection.rb:283:in `put'
/app/vendor/bundle/ruby/3.1.0/gems/aspose_words_cloud-22.9.0/lib/aspose_words_cloud/api_client.rb:154:in `build_request'
/app/vendor/bundle/ruby/3.1.0/gems/aspose_words_cloud-22.9.0/lib/aspose_words_cloud/api_client.rb:80:in `call_api'
/app/vendor/bundle/ruby/3.1.0/gems/aspose_words_cloud-22.9.0/lib/aspose_words_cloud/api/words_api.rb:6478:in `execute_mail_merge_online_with_http_info'
/app/vendor/bundle/ruby/3.1.0/gems/aspose_words_cloud-22.9.0/lib/aspose_words_cloud/api/words_api.rb:6426:in `execute_mail_merge_online'
/app/lib/tape.rb:36:in `merge'
/app/app/jobs/update_instruction_report_job.rb:26:in `update_report'
/app/app/jobs/update_instruction_report_job.rb:12:in `perform'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/execution.rb:65:in `block in _perform_job'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/app/app/jobs/application_job.rb:8:in `block (2 levels) in <class:ApplicationJob>'
/app/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8.7/lib/active_record/connection_adapters/abstract/query_cache.rb:79:in `uncached'
/app/vendor/bundle/ruby/3.1.0/gems/activerecord-7.0.8.7/lib/active_record/query_cache.rb:21:in `uncached'
/app/app/jobs/application_job.rb:7:in `block in <class:ApplicationJob>'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in `instance_exec'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in `block in run_callbacks'
/app/vendor/bundle/ruby/3.1.0/gems/i18n-1.14.6/lib/i18n.rb:353:in `with_locale'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in `instance_exec'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in `block in run_callbacks'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in `instance_exec'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in `block in run_callbacks'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:138:in `run_callbacks'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/execution.rb:64:in `_perform_job'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/instrumentation.rb:20:in `_perform_job'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/execution.rb:52:in `perform_now'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/instrumentation.rb:14:in `block in perform_now'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/instrumentation.rb:25:in `block in instrument'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/notifications.rb:206:in `block in instrument'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
/app/vendor/bundle/ruby/3.1.0/gems/sentry-rails-5.5.0/lib/sentry/rails/tracing.rb:54:in `instrument'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/notifications.rb:206:in `instrument'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/instrumentation.rb:35:in `instrument'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/instrumentation.rb:14:in `perform_now'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/logging.rb:18:in `block in perform_now'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/tagged_logging.rb:99:in `block in tagged'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/tagged_logging.rb:37:in `tagged'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/tagged_logging.rb:99:in `tagged'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/logging.rb:25:in `tag_logger'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/logging.rb:18:in `perform_now'
/app/vendor/bundle/ruby/3.1.0/gems/sentry-rails-5.5.0/lib/sentry/rails/active_job.rb:6:in `perform_now'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/execution.rb:30:in `block in execute'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/railtie.rb:54:in `block (4 levels) in <class:Railtie>'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/execution_wrapper.rb:92:in `wrap'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/reloader.rb:72:in `block in wrap'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/execution_wrapper.rb:92:in `wrap'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/reloader.rb:71:in `wrap'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/railtie.rb:53:in `block (3 levels) in <class:Railtie>'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in `instance_exec'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in `block in run_callbacks'
/app/vendor/bundle/ruby/3.1.0/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:138:in `run_callbacks'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/execution.rb:28:in `execute'
/app/vendor/bundle/ruby/3.1.0/gems/activejob-7.0.8.7/lib/active_job/queue_adapters/resque_adapter.rb:47:in `perform'

nikhil.goyal · February 25, 2025, 11:52am

Why are there so many issues with your servers so frequently.

nikhil.goyal · February 25, 2025, 12:28pm

Please provide the reason as to why it keeps happening

yaroslaw.ekimov · February 26, 2025, 11:04am

It was a network issue that was resolved yesterday. Sorry for the inconvenience.

workflowmax2 · February 26, 2025, 8:50pm

Seems like this issue is back again? @Ivanov_John @yaroslaw.ekimov

aspose.heffron.au · February 27, 2025, 3:21am

We’re having weird connection issues too where the connection is improperly closed by aspose servers.

Also auth doesn’t seem to be working some of the time, it rejects the “grant_type”:“client_credentials” and says it is unknown??

We switched to self hosted to avoid these issues, but looks like now the issue is with the billing API’s so even self hosted gets affected. It was also saying we were unlicenced, just rebooting the container fixed that at least but still can’t generate mail merged documents.

yaroslaw.ekimov · February 27, 2025, 5:25am

I see everything is working now. Sorry for the inconvenience. I asked the responsible team to share why this happened again.

nikhil.goyal · March 3, 2025, 6:42am

@yaroslaw.ekimov please reply regarding why this happened. Its always like that with aspose. You guys provide no further explanation. I have asked this earlier as well

yaroslaw.ekimov · March 3, 2025, 7:47am

I asked the responsible team about an update, and they will share their RCA today.

yaroslaw.ekimov · March 5, 2025, 7:47am

Here is a complete statement from the billing team.
The investigation was concluded, and we determined that a very large number of failed API calls to the /connect/token endpoint was likely the cause of the issue. To mitigate this, we implemented a rate limiter: if a user makes more than 20 failed POST requests to /connect/token within a minute, they will be temporarily blocked from accessing the endpoint for a cooldown period of 10 minutes. When a request is rejected, the system responds with a 429 Too Many Requests status and includes a Retry-After header.

nikhil.goyal · March 12, 2025, 8:15am

@yaroslaw.ekimov we are seeing more errors. What keeps happening so frequently?
its giving us 503 error

nikhil.goyal · March 12, 2025, 8:30am

Simple pdf reports are taking ~90s which took less than 5 sec

yaroslaw.ekimov · March 12, 2025, 9:28am

I asked the admin team to look it ASAP.