Skip to main content

How to delete top n line from flat file using SSIS

Problem: Suppose we have a folder where flat file downloaded from any organisation. Now we have to transfer data from flat file (comma separated file) to our database’s table but top n lines are description about file, this thing may create a problem to access data from flat file to table.File looks like below












Solution: If we remove top n description line from flat file then we can easily process data from flat file to database table.  
Steps -1: Hope you are aware with Flow Control and Data Flow
Step -2: Select tool Script Component













Step -3: Create variable for file path and number of line to delete.


Step-4: Edit Script--> Custom Property--> Read Write Variable and add both user defined variables.


















Step -5: Click on button Edit Script. A script file will be open. Add following name space
using System.IO;
using System.Linq;

Step-6: Write down following code.


    public override void PreExecute()
    {
        base.PreExecute();
      
    }

    /// <summary>
    /// This method is called after all the rows have passed through this component.
    ///
    /// You can delete this method if you don't need to do anything here.
    /// </summary>
    public override void PostExecute()
    {
        string fPath = Variables.fPath;
        int deleteLines = Variables.lineNumber;
        string[] lines = File.ReadAllLines(fPath);
        lines = lines.Skip(deleteLines).ToArray();
        using (StreamWriter sr = new StreamWriter(fPath))
        {
            foreach (var v in lines)
            {
                sr.WriteLine(v);
            }
        }
        base.PostExecute();
        /*
         * Add your code here
         */
    }

    public override void CreateNewOutputRows()
    {
     
         Output0Buffer.AddRow();
         Output0Buffer.MyColumn = 10;
       
    }



Step-7: After saving script run the package you find expected result.








Note: Above script override the existing file. This tutorial is written for SSIS 2012.


Popular posts from this blog

Add day to ISODate in MongoDB

We can use $add operator to add days in ISODate in mongodb, $add is the Arithmetic Aggregation Operator which adds number and date in mongodb.
Syntax:

{ $add: [ <expression1>, <expression2>, ... ] }

Note:  If one of the argument is date $add operator treats to other arguments as milliseconds to add to the date.
Example: Suppose we have a Test collection as below.

{"Title" : "Add day to ISODate in MongoBD","CreatedDate" : ISODate("2016-07-07T08:00:00.000Z")}

Query to add 2 days in CreatedDate

db.Test.aggregate([      { $project: { Title: 1, AddedDate: { $add: [ "$CreatedDate", 2*24*60*60000 ] } } }    ])

Result:

{ "_id" : ObjectId("579a1567ac1b3f3732483de0"), "Title" : "Add day to ISODate in MongoBD", "AddedDate" : ISODate("2016-07-09T08:00:00.000Z") }

Note: As mentioned in above note we have to convert days in millisecond because $add operator treat to other arg…

Remove special characters from string in SQL server

I faced many times an issue to remove special characters from a string. Suppose you are working on searching concept and you have to remove the special characters from search string due to query performance, there are many solution are available but T-SQL is easily resolved this issue.
Following query may help you to resolve your issue.

DECLARE@strVARCHAR(400) DECLARE@expresVARCHAR(50)='%[~,@,#,$,%,&,*,(,),.,!]%' SET@str='(remove) ~special~ *characters. from string in sql!' WHILEPATINDEX(@expres,@str)> 0 BEGIN SET@str=Replace(REPLACE(@str,SUBSTRING(@str,PATINDEX(@expres,@str), 1 ),''),'-',' ') END SELECT@str



What is difference between UNION and UNION ALL in SQL Server

We use UNION and UNION ALL operator to combine multiple results set into one result set.
UNION operator is used to combining multiple results set into one result set but removes any duplicate rows. Basically, UNION is used to performing a DISTINCT operation across all columns in the result set. UNION operator has the extra overhead of removing duplicate rows and sorting result.
UNION ALL operator use to combine multiple results set into one result set but it does not remove any duplicate result. Actually, this does not remove duplicate rows so it is faster than the UNION operator. If you want to combine multiple results and without duplicate records then use UNION otherwise UNION ALL is better.
Following some rules for using UNION/UNION ALL operator
1.The number of the column should be the same in the query's when you want to combine them. 2.The column should be of the same data type. 3.ORDER BY clause can be applied to the overall result set not within each result set.
4.Column name of …