Skip to main content

How to delete top n line from flat file using SSIS

Problem: Suppose we have a folder where flat file downloaded from any organisation. Now we have to transfer data from flat file (comma separated file) to our database’s table but top n lines are description about file, this thing may create a problem to access data from flat file to table.File looks like below












Solution: If we remove top n description line from flat file then we can easily process data from flat file to database table.  
Steps -1: Hope you are aware with Flow Control and Data Flow
Step -2: Select tool Script Component













Step -3: Create variable for file path and number of line to delete.


Step-4: Edit Script--> Custom Property--> Read Write Variable and add both user defined variables.


















Step -5: Click on button Edit Script. A script file will be open. Add following name space
using System.IO;
using System.Linq;

Step-6: Write down following code.


    public override void PreExecute()
    {
        base.PreExecute();
      
    }

    /// <summary>
    /// This method is called after all the rows have passed through this component.
    ///
    /// You can delete this method if you don't need to do anything here.
    /// </summary>
    public override void PostExecute()
    {
        string fPath = Variables.fPath;
        int deleteLines = Variables.lineNumber;
        string[] lines = File.ReadAllLines(fPath);
        lines = lines.Skip(deleteLines).ToArray();
        using (StreamWriter sr = new StreamWriter(fPath))
        {
            foreach (var v in lines)
            {
                sr.WriteLine(v);
            }
        }
        base.PostExecute();
        /*
         * Add your code here
         */
    }

    public override void CreateNewOutputRows()
    {
     
         Output0Buffer.AddRow();
         Output0Buffer.MyColumn = 10;
       
    }



Step-7: After saving script run the package you find expected result.








Note: Above script override the existing file. This tutorial is written for SSIS 2012.


Popular posts from this blog

Check for changes to an SQL Server table?

Problem: Suppose your team is working on the under-development project so it might be possible continuous work on the database and perform changes in Table, Stored procedure as per requirement, and daily you have to update the testing server database as per changes are done in developing server database then how it is possible to trace those changes. There are a lot of solutions for this problem which is listed below Solution 1: For SQL Server 2000, 2005 and above use the CHECKSUM command SELECTCHECKSUM_AGG(BINARY_CHECKSUM(*))FROMYour_Table_NameWITH (NOLOCK); That will return the same number each time its run as long as the table contents haven't changed. Unfortunately CHECKSUM does not work always properly to detect changes. It is only a primitive checksum and no CRC calculation. Therefore you can't use it to detect all changes, e. g. symmetrical changes result in the same CHECKSUM! Solution 2: 1.Run the following query. Before executing query replace DB_Name with your database name…

How to drop multiple tables with common prefix in one query?

Problem: Suppose we have a situation where we need to drop those tables that have some prefixes string, so it is possible to drop those tables with a common prefix in a single query.
Solution: yes it is possible to drop all those tables that have the common prefix in a single query. Using following query you can delete all those tables that begin with a certain prefix string. In where condition just put the common prefix string in where condition (Like ‘prefix%’)
DECLARE@queryNVARCHAR(MAX)=N'';
SELECT@query+=' DROP TABLE ' +QUOTENAME(s.name) +'.'+QUOTENAME(t.name)+';' FROMsys.tablesASt INNERJOINsys.schemasASs ONt.[schema_id]=s.[schema_id] WHEREt.nameLIKE'MX_100%';
EXECsp_executesql@query;
This query may create an issue, if a table has a foreign key relationship, you'll either need to drop them first or arrange the output to drop the tables in a certain order. If you want to monitor exactly what goes on when the query is running then use the following que…

Merge and Merge join transformation in SSIS