Fivetran: Syncs not running per schedule for AWS us-east-1 region
Incident
Report for Fivetran
Resolved
This incident has been resolved, and we have observed that affected connections are now syncing successfully.
Incident Summary
Description:
Connections associated with destinations using the AWS us-east-1 region as a cloud service provider were unable to run syncs.
Timeline:
This issue began on September 24th at 15:24 UTC and was resolved on September 24th at 18:53 UTC.
Cause:
More connections were attempting to be established than our client connection pool allowed, causing pipeline delays and eventually sync start failures.
Resolution:
We mitigated this issue by increasing the client connection pool in the region, allowing for more concurrent connections.
Incident Summary
Description:
Connections associated with destinations using the AWS us-east-1 region as a cloud service provider were unable to run syncs.
Timeline:
This issue began on September 24th at 15:24 UTC and was resolved on September 24th at 18:53 UTC.
Cause:
More connections were attempting to be established than our client connection pool allowed, causing pipeline delays and eventually sync start failures.
Resolution:
We mitigated this issue by increasing the client connection pool in the region, allowing for more concurrent connections.
Monitoring
A fix has been deployed and we are observing affected connections returning to normal operation.
Identified
Some connections may show "delayed" status and may fail to start syncs per their schedule.
Affected connections are associated with destinations using the AWS us-east-1 region as a cloud service provider.
We are actively investigating this issue.
Affected connections are associated with destinations using the AWS us-east-1 region as a cloud service provider.
We are actively investigating this issue.
This incident affected: Systems (Transformations), Database connectors (Amazon Aurora MySQL, Amazon Aurora PostgreSQL, Azure Database for MariaDB, Azure Database for MySQL, Azure Database for PostgreSQL, Azure SQL Database, Azure SQL Managed Instance, Amazon DynamoDB, Google Cloud SQL for MySQL, Google Cloud SQL for PostgreSQL, Google Cloud SQL for SQL Server, Heroku PostgreSQL, Magento MySQL, Magento MySQL RDS, MariaDB, Amazon RDS for MariaDB, MongoDB, MongoDB Sharded, MySQL, MySQL RDS, Oracle, Oracle EBS, Oracle RAC, Oracle RDS, PostgreSQL, Amazon RDS for PostgreSQL, SAP HANA, SQL Server, Amazon RDS for SQL Server, Amazon DocumentDB, High-Volume Agent Oracle, High Volume Agent Db2 for i, High-Volume Agent SQL Server, Snowflake, High-Volume Agent SAP ECC on Oracle, Elastic Cloud, Self Hosted Elasticsearch, Open Distro, Opensearch, High-Volume Agent SAP ECC on Oracle with NetWeaver, Azure Cosmos DB for NoSQL, High-Volume Agent SAP ECC on Db2 for i, HVA SAP ECC on SQL Server, BigQuery, Firebase, Azure CosmosDB for MongoDB, SAP ERP on HANA, Convex, PlanetScale), Finance connectors (Anaplan, Braintree, Braintree Sandbox, Coupa, Lightspeed Retail, NetSuite SuiteAnalytics, QuickBooks, ReCharge, Recurly, Sage Intacct, SAP Concur, SAP S/4, Square, Stripe, Stripe test mode, Xero, Zuora, Zuora Sandbox, Salesforce Commerce Cloud, SAP Business ByDesign, Chargebee Product Catalog 1.0, Chargebee Product Catalog 2.0, Paypal, Paypal Sandbox, Subscript, GoCardless, Oracle Fusion Cloud Applications - FSCM(ERP & SCM), Open Exchange Rates, EasyPost, Brex, Squarespace, FreshBooks, ShipHero, Ramp, Revel Systems, Flexport, Navan, Afterpay, Maxio Chargify, EZOfficeInventory, Checkout.com, Klarna, Standard Metrics, Maxio SaaSOptics, Sharetribe, Rithum, Younium, Adyen, Mambu, Aumni, Zoho Books, Column, Cin7 Core, AWS Cost Explorer, Shopware, Microsoft Dynamics 365 Finance and Operations, Expensify, eBay, ReBound Returns, Microsoft Dynamics 365 Business Central, Workday Financial Management, Anvyl, Ordway, RevenueCat), Sales connectors (Microsoft Dynamics 365 CRM, Outreach, Pipedrive, Salesforce, Salesforce sandbox, Shopify, Big Commerce, Crossbeam, Salesloft, Oracle Fusion Cloud Applications - CRM, Close, Microsoft Dynamics 365 Finance, Amazon Selling Partner, Clari, Duoplane, loop, HubSpot, Copper, Oracle Business Intelligence Publisher, Lemlist, DoorLoop, Ricochet360, Veeqo, Boostr, Attio, Buildium, RevOps), Event connectors (Amplitude, Apache Kafka, AWS MSK, Azure Event Hubs, Confluent Cloud, Heap, Heroku Kafka, Mixpanel, Optimizely, Pendo, Segment, Snowplow, Webhooks, Azure Service Bus), Function connectors (AWS Lambda, Azure Functions, Google Cloud Functions), Marketing connectors (Adobe Analytics, Adobe Analytics Data Feed, AdRoll, Apple App Store, Apple Search Ads, AppsFlyer, Branch, Braze, Criteo, Eloqua, Facebook Ads, Facebook Pages, Google Ad Manager, Google Analytics, Google Analytics 360, Google Analytics (MCF), Google Campaign Manager 360, Google Display & Video 360, Google Play, Google Search Console, Instagram Business, Iterable, Klaviyo, LinkedIn Ad Analytics, LinkedIn Company Pages, Mailchimp, Mandrill, Marin Software, Marketo, Microsoft Advertising, Outbrain, Pardot, Pinterest Ads, Sailthru, Salesforce Marketing Cloud, SendGrid, Snapchat Ads, Taboola, Tiktok Ads, Twitter Organic, Twitter Ads, Verizon Media, YouTube Analytics, Google Analytics 4 Export, Google Analytics 4, Adjust, Google Ads, Amazon Ads, ActiveCampaign, Reddit Ads, WooCommerce, Aircall, ChartMogul, The Trade Desk, Lob, Impact, Chorus, Mailgun, Orbit, Retently, Drip, RetailNext, Campaign Monitor, Eventbrite, Mux, Kissmetrics, ChargeDesk, Google Search Ads 360, Churnkey, commercetools, Ortto, StackAdapt, Destini, Luma, Upland Software, Integral Ad Science, Mention, Awin, Trisolute, PagerDuty, MNTN, Zoho Campaigns, Stylight, BallotReady, Customer.io, Mixmax, Hopin, Skilljar, Smadex, Forj Community, JustCall, Katana, Ometria, Attentive, Brevo, PartnerStack Vendor, PowerReviews Enterprise, UpPromote, Vitally, iLevel, Amazon Attribution, Prive, ironSource, Reply.io, Oracle Moat Analytics, Piwik PRO, Pinterest Organic, Bitly, Integrate, Moloco, Affinity, RTB House, Sprout Social, CJ Commission Detail, GRIN, Postmark, Rakuten Advertising, Google Business Profile, ON24, Webconnex, Brightcove, Megaphone, REVIEWS.io, SparkPost, Trelica, VWO, Liftoff, ReferralHero, Spotify Ads, Electronic Tenant Solutions, Short.io, Simplecast, Partnerize, Quora Ads, Swoogo, Brave Ads, Eventsforce, Teads, YouGov Sport, Fone Dynamics, Ticket Tailor, Kevel, Xandr, Bizzabo, RevX, Bazaarvoice), Human Resources connectors (ADP Workforce Now, Greenhouse, Lever, Workday RaaS, Workday HCM, Lattice, Bamboo HR, Oracle Fusion Cloud Applications - HCM, Lessonly, Fountain, 360Learning, SCORM, When I Work, Docebo, Workable, FreshTeam, Papershift, HiBob, HiBob, Gem, Employment Hero, 15Five, Namely, Security Journey, Rippling, Deputy, Care Quality Commission, Zenefits, Persona, Personio, WorkRamp, KeyPay, Pinpoint, Thinkific, Paychex, UKG Pro, Factorial, Culture Amp, Udemy Business, 7shifts, Proofpoint Security Awareness, SkillsTX, Tymeshift, Absorb LMS, Teamtailor, SmartRecruiters, SmartHR), Productivity connectors (Airtable, Asana, Height, Kantata, ServiceNow, Drift, Open Data DC, SonarQube, Gainsight Customer Success, Ironclad, Gong, Slack, Freightview, Okta, Calendly, Wrike, Harvest, Cin7, ShipStation, Samsara, Dixa, SkuVault, Friendbuy, Google Calendar, Rootly, ChurnZero, DCL Logistics, Wherefour, Toggl Track, Insightly, Loopio, Buzzsprout, GlobalMeet, Float, Smartwaiver, Jotform, WordPress, Castor EDC, HappyFox, Dropbox Sign, Yahoo Search Ads on Yahoo Japan, Concord, LinkSquares, Zoom, Microsoft Entra ID, Checkr, LiveChat Partner, VTS, Rocketlane, StarRez, CallRail, Azure DevOps, Picqer, Pivotal Tracker, Talkwalker, Productive, OnceHub, ShipNetwork, Gmail), File connectors (Amazon CloudFront, Amazon S3, AWS CloudTrail, Azure Blob Storage, Box, Dropbox, Email, FTP, Google Cloud Storage, Google Drive, Google Sheets, OneDrive, SFTP, SharePoint, Amazon Kinesis Firehose), Log connectors (AWS CloudWatch, Datadog Log, Fivetran Platform, Google Cloud Logging, Azure Monitor Log, Splunk Log, Newrelic Log), Engineering connectors (GitHub, Jira, Splunk, Twilio, Microsoft Lists, Buildkite, Rarible, Freshservice, Statsig, Split, Monday.com, Rollbar, Chameleon, ClickUp, Productboard, Guru, Tempo, Collibra, PostHog, Trello, Opsgenie, Notion, FullStory, Teamwork, Appfigures, Everhour, Kandji, Salsify, Tableau, LaunchDarkly, Appcues, CircleCI, Canny, dbt Cloud, Looker, Calabrio, Linear, Xray, Statuspage, Learnupon, Incident.io, crowd.dev, Aha!, Codefresh, Shortcut, Cvent, Jira Align, Adobe Workfront, GitLab, Delighted, TestRail, FireHydrant, SolarWinds Service Desk, Confluence, Mode, SafetyCulture, Pingdom, Sigma Computing, Factbird, Resource Management by Smartsheet), Security connectors (Auth0, Kisi, Transcend), and Support connectors (Freshdesk, Front, Help Scout, Intercom, Kustomer, Qualtrics, SurveyMonkey, Typeform, UserVoice, Zendesk Chat, Zendesk Sell, Zendesk Support, Zendesk Sunshine, Zoho CRM, Medallia, RingCentral, Assembled, Alchemer, Khoros Care, Ada, Survicate, Talkdesk, Gorgias, Gladly, Freshchat, Birdeye, Qualaroo, Simplesat, Playvox, GetFeedback, Freshdesk Contact Center, Helpshift, Zoho Desk, Ivanti, DSV, Genesys, Ringover, ServiceTitan, Vonage Contact Center, Delighted, Medallia Agile Research).