{ "id": "sql-server", "title": "Prepare SQL Server for RDI", "url": "https://redis.io/docs/latest/integrate/redis-data-integration/data-pipelines/prepare-dbs/sql-server/", "summary": "Prepare SQL Server databases to work with RDI", "tags": [ "docs", "integrate", "rs", "rdi" ], "last_updated": "2026-04-01T08:10:08-05:00", "page_type": "content", "content_hash": "bbaed0c491ba229ec478c72973b5778d64bb56440a015bcc7db45c0933612932", "sections": [ { "id": "overview", "title": "Overview", "role": "overview", "text": "To prepare your SQL Server database for Debezium, you must first create a dedicated Debezium user,\nrun a script to enable CDC globally, and then separately enable CDC for each table you want to\ncapture. You need administrator privileges to do this.\n\nOnce you enable CDC, it captures all of the INSERT, UPDATE, and DELETE operations\non your chosen tables. The Debezium connector can then emit these events to RDI.\n\nThe following checklist summarizes the steps to prepare a SQL Server\ndatabase for RDI, with links to the sections that explain the steps in\nfull detail. You may find it helpful to track your progress with the\nchecklist as you complete each step.\n\n[code example]" }, { "id": "1-create-a-debezium-user", "title": "1. Create a Debezium user", "role": "content", "text": "It is strongly recommended to create a dedicated Debezium user for the connection between RDI\nand the source database. When using an existing user, ensure that the required \npermissions are granted and that the user is added to the CDC role.\n\n[code example]\n\n1. \n Create the Debezium user with the Transact-SQL below:\n\n [code example]\n\n Replace `MyUser`, `My_Password` and `MyDB` with your chosen values.\n\n1. \n Grant the user the necessary permissions:\n\n [code example]" }, { "id": "2-enable-cdc-on-the-database", "title": "2. Enable CDC on the database", "role": "content", "text": "There are two system stored procedures to enable CDC (you need\nadministrator privileges to run these). Use `sys.sp_cdc_enable_db`\nto enable CDC for the whole database and then `sys.sp_cdc_enable_table` to enable CDC for individual tables. \n\nBefore running the procedures, ensure that:\n\n- You are a member of the `sysadmin` fixed server role for the SQL Server.\n- You are a `db_owner` of the database.\n- The SQL Server Agent is running.\n\nThen, assuming your database is called `MyDB`, run the script below to enable CDC:\n\n[code example]\n\nFor SQL Server on AWS RDS, you must use a different stored procedure:\n[code example]\n\n\nWhen you enable CDC for the database, it creates a schema called `cdc` and also\na CDC user, metadata tables, and other system objects." }, { "id": "3-enable-cdc-for-the-tables-you-want-to-capture", "title": "3. Enable CDC for the tables you want to capture", "role": "content", "text": "[code example]\n\n1. \n You must also enable CDC on the tables you want Debezium to capture using the\n following commands (again, you need administrator privileges for this):\n\n [code example]\n\n Repeat this for every table you want to capture.\n\n The value for `@role_name` can’t be a fixed database role, such as `db_datareader`. \n Specifying a new name will create a corresponding database role that has full access to the\n captured change data.\n \n \n1. \n Add the Debezium user to the CDC role:\n\n [code example]" }, { "id": "4-check-that-you-have-access-to-the-cdc-table", "title": "4. Check that you have access to the CDC table", "role": "content", "text": "You can use another stored procedure `sys.sp_cdc_help_change_data_capture`\nto query the CDC information for the database and check you have enabled\nit correctly. To do this, connect as the Debezium user you created previously (`MyUser`).\n\n[code example]\n\n1. \n Run the `sys.sp_cdc_help_change_data_capture` stored procedure to query\n the CDC configuration. For example, if your database was called `MyDB` then you would\n run the following:\n\n [code example]\n\n1. \n The query returns configuration information for each table in the database that\n has CDC enabled and that contains change data that you are authorized to\n access. If the result is empty then you should check that you have privileges\n to access both the capture instance and the CDC tables." }, { "id": "troubleshooting", "title": "Troubleshooting", "role": "errors", "text": "If no CDC is happening then it might mean that SQL Server Agent is down. You can check for this using the SQL query shown below:\n\n[code example]\n\nIf the query returns a result of 0, you need to need to start SQL Server Agent using the following commands:\n\n[code example]" }, { "id": "sql-server-capture-job-agent-configuration-parameters", "title": "SQL Server capture job agent configuration parameters", "role": "content", "text": "In SQL Server, the parameters that control the behavior of the capture job agent\nare defined in the SQL Server table `msdb.dbo.cdc_jobs`. If you experience performance\nproblems while running the capture job agent then you can adjust the capture jobs\nsettings to reduce CPU load. To do this, run the `sys.sp_cdc_change_job` stored procedure\nwith your new parameter values.\n\nA full guide to configuring the SQL Server capture job agent parameters\nis outside the scope of the Redis documentation.\n\nThe following parameters are the most important ones for modifying the capture agent behavior\nof the Debezium SQL Server connector:\n\n* `pollinginterval`: This specifies the number of seconds that the capture agent\n waits between log scan cycles. A higher value reduces the load on the database\n host, but increases latency. A value of 0 specifies no wait between scans.\n The default value is 5.\n* `maxtrans`: This specifies the maximum number of transactions to process during\n each log scan cycle. After the capture job processes the specified number of\n transactions, it pauses for the length of time that `pollinginterval` specifies\n before the next scan begins. A lower value reduces the load on the database host,\n but increases latency. The default value is 500.\n* `maxscans`: This specifies a limit on the number of scan cycles that the capture\n job can attempt when capturing the full contents of the database transaction log.\n If the continuous parameter is set to 1, the job pauses for the length of time\n that the `pollinginterval` specifies before it resumes scanning. A lower values\n reduces the load on the database host, but increases latency. The default value is 10.\n\nSee the SQL Server documentation for more information about capture agent parameters." }, { "id": "sql-server-on-azure", "title": "SQL Server on Azure", "role": "content", "text": "You can also use the Debezium SQL Server connector with SQL Server on Azure.\nSee Microsoft's guide to\n[configuring SQL Server on Azure for CDC with Debezium](https://learn.microsoft.com/en-us/samples/azure-samples/azure-sql-db-change-stream-debezium/azure-sql%2D%2Dsql-server-change-stream-with-debezium/)\nfor more information." }, { "id": "handling-changes-to-the-schema", "title": "Handling changes to the schema", "role": "content", "text": "RDI can't adapt automatically when you change the schema of a CDC table in SQL Server. For example,\nif you add a new column to a table you are capturing then RDI will generate errors\ninstead of capturing the changes correctly. See Debezium's\n[SQL Server schema evolution](https://debezium.io/documentation/reference/stable/connectors/sqlserver.html#sqlserver-schema-evolution)\ndocs for more information.\n\nIf you have administrator privileges, you can follow the steps below to update RDI after\na schema change and resume CDC. See the\n[online schema updates](https://debezium.io/documentation/reference/stable/connectors/sqlserver.html#online-schema-updates)\ndocumentation for further details.\n\n[code example]\n\n1. \n Make your changes to the source table schema.\n\n1. \n Create a new capture table for the updated source table by running the `sys.sp_cdc_enable_table` stored\n procedure with a new, unique value for the parameter `@capture_instance`. For example, if the old value\n was `dbo_MyTable`, you could replace it with `dbo_MyTable_v2` (you can see the existing values by running\n stored procedure `sys.sp_cdc_help_change_data_capture`):\n\n [code example]\n\n1. \n When Debezium starts streaming from the new capture table, drop the old capture table by running \n the `sys.sp_cdc_disable_table` stored procedure with the parameter `@capture_instance` set to the old\n capture instance name, `dbo_MyTable`:\n\n [code example]\n\nRDI will *not* correctly capture changes that happen in the time gap between changing\nthe source schema (step 1 above) and updating the value of `@capture_instance` (step 2).\nTry to keep the gap as short as possible or perform the update at a time when you expect\nfew changes to the data." } ], "examples": [ { "id": "overview-ex0", "language": "checklist {id=\"sqlserverlist\"}", "code": "- [ ] [Create a Debezium user](#1-create-a-debezium-user)\n- [ ] [Enable CDC on the database](#2-enable-cdc-on-the-database)\n- [ ] [Enable CDC for the tables you want to capture](#3-enable-cdc-for-the-tables-you-want-to-capture)\n- [ ] [Check that you have access to the CDC table](#4-check-that-you-have-access-to-the-cdc-table)", "section_id": "overview" }, { "id": "1-create-a-debezium-user-ex0", "language": "checklist {id=\"sqlserver-create-debezium-user\" nointeractive=\"true\" }", "code": "- [ ] [Create the Debezium user](#create-the-debezium-user)\n- [ ] [Grant the user the necessary permissions](#grant-the-user-the-necessary-permissions)", "section_id": "1-create-a-debezium-user" }, { "id": "1-create-a-debezium-user-ex1", "language": "sql", "code": "USE master\n GO\n CREATE LOGIN MyUser WITH PASSWORD = 'My_Password'\n GO\n USE MyDB\n GO\n CREATE USER MyUser FOR LOGIN MyUser\n GO", "section_id": "1-create-a-debezium-user" }, { "id": "1-create-a-debezium-user-ex2", "language": "sql", "code": "USE master\n GO\n GRANT VIEW SERVER STATE TO MyUser\n GO\n USE MyDB\n GO\n EXEC sp_addrolemember N'db_datareader', N'MyUser'\n GO", "section_id": "1-create-a-debezium-user" }, { "id": "2-enable-cdc-on-the-database-ex0", "language": "sql", "code": "USE MyDB\nGO\nEXEC sys.sp_cdc_enable_db\nGO", "section_id": "2-enable-cdc-on-the-database" }, { "id": "2-enable-cdc-on-the-database-ex1", "language": "sql", "code": "EXEC msdb.dbo.rds_cdc_enable_db 'Chinook'\nGO", "section_id": "2-enable-cdc-on-the-database" }, { "id": "3-enable-cdc-for-the-tables-you-want-to-capture-ex0", "language": "checklist {id=\"sqlserver-enable-cdc-tables\" nointeractive=\"true\" }", "code": "- [ ] [Enable CDC on the tables you want to capture](#enable-cdc-on-the-tables-you-want-to-capture)\n- [ ] [Add the Debezium user to the CDC role](#add-the-debezium-user-to-the-cdc-role)", "section_id": "3-enable-cdc-for-the-tables-you-want-to-capture" }, { "id": "3-enable-cdc-for-the-tables-you-want-to-capture-ex1", "language": "sql", "code": "USE MyDB\n GO\n\n EXEC sys.sp_cdc_enable_table\n @source_schema = N'dbo',\n @source_name = N'MyTable', \n @role_name = N'MyRole', \n @supports_net_changes = 0\n GO", "section_id": "3-enable-cdc-for-the-tables-you-want-to-capture" }, { "id": "3-enable-cdc-for-the-tables-you-want-to-capture-ex2", "language": "sql", "code": "USE MyDB\n GO\n EXEC sp_addrolemember N'MyRole', N'MyUser'\n GO", "section_id": "3-enable-cdc-for-the-tables-you-want-to-capture" }, { "id": "4-check-that-you-have-access-to-the-cdc-table-ex0", "language": "checklist {id=\"sqlserver-check-cdc-table\" nointeractive=\"true\" }", "code": "- [ ] [Run the stored procedure to query the CDC configuration](#run-the-stored-procedure-to-query-the-cdc-configuration)\n- [ ] [Check the results](#check-the-results)", "section_id": "4-check-that-you-have-access-to-the-cdc-table" }, { "id": "4-check-that-you-have-access-to-the-cdc-table-ex1", "language": "sql", "code": "USE MyDB;\n GO\n EXEC sys.sp_cdc_help_change_data_capture\n GO", "section_id": "4-check-that-you-have-access-to-the-cdc-table" }, { "id": "troubleshooting-ex0", "language": "sql", "code": "IF EXISTS (SELECT 1 \n FROM master.dbo.sysprocesses \n WHERE program_name = N'SQLAgent - Generic Refresher')\nBEGIN\n SELECT @@SERVERNAME AS 'InstanceName', 1 AS 'SQLServerAgentRunning'\nEND\nELSE \nBEGIN\n SELECT @@SERVERNAME AS 'InstanceName', 0 AS 'SQLServerAgentRunning'\nEND", "section_id": "troubleshooting" }, { "id": "troubleshooting-ex1", "language": "sql", "code": "EXEC xp_servicecontrol N'START',N'SQLServerAGENT';\nGO", "section_id": "troubleshooting" }, { "id": "handling-changes-to-the-schema-ex0", "language": "checklist {id=\"sqlserver-schema-changes\" nointeractive=\"true\" }", "code": "- [ ] [Make your changes to the source table schema](#make-your-changes-to-the-source-table-schema)\n- [ ] [Create a new capture table for the updated source table](#create-a-new-capture-table-for-the-updated-source-table)\n- [ ] [Drop the old capture table](#drop-the-old-capture-table)", "section_id": "handling-changes-to-the-schema" }, { "id": "handling-changes-to-the-schema-ex1", "language": "sql", "code": "EXEC sys.sp_cdc_enable_table\n @source_schema = N'dbo',\n @source_name = N'MyTable',\n @role_name = N'MyRole',\n @capture_instance = N'dbo_MyTable_v2',\n @supports_net_changes = 0\n GO", "section_id": "handling-changes-to-the-schema" }, { "id": "handling-changes-to-the-schema-ex2", "language": "sql", "code": "EXEC sys.sp_cdc_disable_table\n @source_schema = N'dbo',\n @source_name = N'MyTable',\n @capture_instance = N'dbo_MyTable'\n GO", "section_id": "handling-changes-to-the-schema" } ] }