Skip to main content

Small enhancement history of SSIS

In SQL Server 7.0 Microsoft worked on Data Transformation Service, there was a small team who worked on DTS features which was used for Import/Export Wizard. Starting purpose was to transfer data using OLEDB from any data source to any destination. It also has the ability to execute programs and run scripts, making workflow a minor feature.

in SQL Server 2000 Microsoft enhance the features  as following.

  1. Dynamic Property Task : Its enable you to alter package dynamically at run time.
  2. ActiveX script: The main purpose of ActiveX script is to Including business logic in packages. For example, ActiveX script can use conditional logic to manage package workflow. 

ActiveX script create some problems for DBA because that time for script they used VBScript but DBA don't have any experience to write VBScript.

After Five years Microsoft release SQL Server 2005 and SSIS, which was have many new features as following.
  1. Import/Export wizard improved for multiple-table scenario.
  2. To change data type added Data Conversion Transformation.
  3. Include Sort Transformation
  4. Advance Editor UseBibaryFormat, CommandTimeout,Expression Builder etc.

But SSIS SQL Server 2005 was not more longer stand in market.

SQL Server 2008 comes with extra scalability features to help enterprise, Microsoft made a huge investment in usability, with simple enhancement like
  1. Toolbox: That's make easier to use.
  2. Focused on management and deployment.  
  3. Lookup Enhancement
  4. Data Profiling Task
  5. Pipeline Memory Limiter
  6. Pipeline Performance – Multicast Transform and Threading in SQL Server 2008
  7. VSTA support for Script Task and Script Component
  8. Import & Export Wizard Enhancements
  9. Brief explanation of leveraging SQL Server 2008 features in SSIS viz. CDC and MERGE statement

After Release of SQL Server 2008 Microsoft enhance the features in SSIS SQL Server 2012. Microsoft focused on SSIS manageability and making easier to deployment.
  1. SQL Server 2012 added ROBUST new data cleaning components that helps you standardize and detect data anomalies.
  2. Deployment
  3. Management and Troubleshooting
  4. Development Enhancements
  5. Performance
  6. Data Quality
  7. Access to Samples and Tutorials 

There was no major enhancement comes to SSIS 2014 It's all features near about same as SSIS 2012. But many enhancement and new features comes in SSIS SQL Server 2016.

1- Manageability
         a) Better deployment
                   1- SSISDB Upgrade Wizard
                   2- Support for Always On in the SSIS Catalog
                   3- Incremental package deployment
                   4- Support for Always Encrypted in the SSIS Catalog
         b) Better debugging
                   1- New ssis_logreader database-level role in the SSIS catalog
                   2- New RuntimeLineage logging level in the SSIS catalog
                   3- New custom logging level in the SSIS catalog
                   4- Column names for errors in the data flow
                   5- Expanded support for error column names
                   6- Support for server-wide default logging level
                   7- New IDTSComponentMetaData130 interface in the API
         c) Better package management
                   1- Improved experience for project upgrade
                   2- AutoAdjustBufferSize property automatically calculates buffer                            size for data flow
                   3- Reusable control flow templates
                   4- New templates renamed as parts
2- Connectivity
         a) Expanded connectivity on premises
                   1- Support for OData v4 data sources
                   2- Explicit support for Excel 2013 data sources
                   3- Support for the Hadoop file system (HDFS)
                   4- Expanded support for Hadoop and HDFS
                   5- HDFS File Destination now supports ORC file format
                   6- ODBC components updated for SQL Server 2016
                   7- Explicit support for Excel 2016 data sources
                   8- Connector for SAP BW for SQL Server 2016 released
                   9- Connectors v4.0 for Oracle and Teradata released
         b) Expanded connectivity to the cloud
                   1- Azure Storage connectors and Hive and Pig tasks for HDInsight                        - Azure Feature Pack for SSIS released for SQL Server 2016
3- Usability and productivity
         a) Better install experience
                 1- Upgrade blocked when SSISDB belongs to an Availability Group
         b) Better design experience
                   1- SSIS Designer creates and maintains packages for SQL Server                          2016, 2014, or 2012
                   2- Multiple designer improvements and bug fixes.
         c)Better management experience in SQL Server Management                     Studio
                   1- Improved performance for SSIS Catalog views
 4- Other enhancements
         1- Balanced Data Distributor transformation is now part of SSIS
         2- Data Feed Publishing Components are now part of SSIS
         3- Support for Azure Blob Storage in the SQL Server Import and Export                  Wizard
         4- Change Data Capture Designer and Service for Oracle for Microsoft                    SQL Server 2016 released
         5- CDC components updated for SQL Server 2016
         6- Analysis Services Execute DDL Task updated
         7- Analysis Services tasks support tabular models
         8- Support for Built-in R Services
         9- Rich XML validation output in the XML Task

Popular posts from this blog

What is difference between UNION and UNION ALL in SQL Server

We use UNION and UNION ALL operator to combine multiple results set into one result set.
UNION operator is used to combining multiple results set into one result set but removes any duplicate rows. Basically, UNION is used to performing a DISTINCT operation across all columns in the result set. UNION operator has the extra overhead of removing duplicate rows and sorting result.
UNION ALL operator use to combine multiple results set into one result set but it does not remove any duplicate result. Actually, this does not remove duplicate rows so it is faster than the UNION operator. If you want to combine multiple results and without duplicate records then use UNION otherwise UNION ALL is better.
Following some rules for using UNION/UNION ALL operator
1.The number of the column should be the same in the query's when you want to combine them. 2.The column should be of the same data type. 3.ORDER BY clause can be applied to the overall result set not within each result set.
4.Column name of …

Disable/Enable All the Foreign Key Constraint in SQL Server Database

In some scenario we need to disable all foreign key relationships from all table, suppose for bad testing purpose or you want to see what will be effect of garbage data on application or performance, due to this purpose you want to insert garbage data into table and this table contains foreign key constraints then you will get the error of foreign key constraint.
Many people face this types of issue, the following query will resolve this problem but be careful on the production server, it would be better if you do this under guidance to any DBA.
The following query will disable all constraints in the database.

execsp_msforeachtable'alter table ? nocheck constraint all'

Once you disabled all constraints you should also need to enable all constraints once you completed your work, so following query will help you to enable all constraints in the database.

execsp_msforeachtable'alter table ? with check check constraint all'

Using the above query you can resolve above described s…

Dropdownlist selectedindexchanged event is not firing

<asp:DropDownList ID="ddlSource" runat="server" DataSourceID="SqlDataSource1" DataTextField="vcSuplierNm" ViewStateMode="Enabled DataValueField="vcSuplierCode" EnableViewState="true" AppendDataBoundItems="true" OnSelectedIndexChanged="ddl_OnSelectedIndexChanged" AutoPostBack="true"></asp:DropDownList>

Add property ViewStateMode="Enabled" and EnableViewState="true"
in drop DropDownList

Add day to ISODate in MongoDB

We can use $add operator to add days in ISODate in mongodb, $add is the Arithmetic Aggregation Operator which adds number and date in mongodb.
Syntax:

{ $add: [ <expression1>, <expression2>, ... ] }

Note:  If one of the argument is date $add operator treats to other arguments as milliseconds to add to the date.
Example: Suppose we have a Test collection as below.

{"Title" : "Add day to ISODate in MongoBD","CreatedDate" : ISODate("2016-07-07T08:00:00.000Z")}

Query to add 2 days in CreatedDate

db.Test.aggregate([      { $project: { Title: 1, AddedDate: { $add: [ "$CreatedDate", 2*24*60*60000 ] } } }    ])

Result:

{ "_id" : ObjectId("579a1567ac1b3f3732483de0"), "Title" : "Add day to ISODate in MongoBD", "AddedDate" : ISODate("2016-07-09T08:00:00.000Z") }

Note: As mentioned in above note we have to convert days in millisecond because $add operator treat to other arg…

Remove special characters from string in SQL server

I faced many times an issue to remove special characters from a string. Suppose you are working on searching concept and you have to remove the special characters from search string due to query performance, there are many solution are available but T-SQL is easily resolved this issue.
Following query may help you to resolve your issue.

DECLARE@strVARCHAR(400) DECLARE@expresVARCHAR(50)='%[~,@,#,$,%,&,*,(,),.,!]%' SET@str='(remove) ~special~ *characters. from string in sql!' WHILEPATINDEX(@expres,@str)> 0 BEGIN SET@str=Replace(REPLACE(@str,SUBSTRING(@str,PATINDEX(@expres,@str), 1 ),''),'-',' ') END SELECT@str



Query to find foreign key relationship and name of the constraints for each table in the database

Sometimes we need to find all foreign key on a table so that we can analyze the relation of a table and can find the solution accordingly. It is very difficult to see it manually, by using the following query we can find all foreign key relationships on the particular table.

SELECT K_Table=FK.TABLE_NAME, FK_Column=CU.COLUMN_NAME, PK_Table=PK.TABLE_NAME, PK_Column=PT.COLUMN_NAME, Constraint_Name=C.CONSTRAINT_NAME FROMINFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTSC INNERJOININFORMATION_SCHEMA.TABLE_CONSTRAINTSFKONC.CONSTRAINT_NAME=FK.CONSTRAINT_NAME INNERJOININFORMATION_SCHEMA.TABLE_CONSTRAINTSPKONC.UNIQUE_CONSTRAINT_NAME=PK.CONSTRAINT_NAME INNERJOININFORMATION_SCHEMA.KEY_COLUMN_USAGECUONC.CONSTRAINT_NAME=CU.CONSTRAINT_NAME INNERJOIN( SELECTi1.TABLE_NAME,i2.COLUMN_NAME FROMINFORMATION_SCHEMA.TABLE_CONSTRAINTSi1 INNERJOININFORMATION_SCHEMA

How to change schema of table in SQL Server

In SQL Server 2005 and above versions  all tables grouped into schemas. When we creating a table without specify a schema SQL Server creates a default schema "dbo".  We can alter the schema of table. For example I am creating a table with default schema see bellow script.
CREATETABLE Codefari (             id INTIDENTITY(1,1),             title VARCHAR(400) ) SELECT name,SCHEMA_NAME(schema_id)as [Schema] FROMsys.tables WHERE  name ='Codefari'
Result set
name            Schema ------------------------------------------------ Codefari       dbo
(1 row(s) affected)
In result set we can see schema is dbo, if we want change schema of table then first we have to create a schema if does not exist. See below script.
CREATESCHEMA Blog GO ALTERSCHEMA Blog TRANSFER dbo.Codefari GO
SELECT name,SCHEMA_NAME(schema_id)as [Schema] FROMsys.tables WHERE  name ='Codefari'

Like in MongoDB Query

In SQL, 'LIKE' query look like as below
SELECT*FROM[dbo.][USERS]WHEREFNAMELIKE'%DILIP%'
In MongoDB it's look like as below(query in MongoDB console)
db.USERS.find({"FNAME":/DILIP/})
Same as
db.USERS.find({"FNAME":/^DILIP/})   // FNAMELIKE'DILIP%'
db.USERS.find({"FNAME":/DILIP$/})   // FNAMELIKE'%DILIP'

$regex in MongoDB