Quantcast
Channel: Programming Forums
Viewing all articles
Browse latest Browse all 51036

How to send datatable to Access

$
0
0
Oh HAI THAR! It's me again. This time I'm updating a MS Access database from a vb.net Data table which is constructed by reading in a number of .csv files with various fields. The data table contains fields which represent the total of all fields across all files (there ends up being something on the order of 155 variables), so the individual files are first read into a table dynamically generated to contain only those field which exist in the table, then those fields are shunted into the master table and matched up with the appropriate field name. After all that, the master data table is sent into an access database using, what may well be, the worst possible method.

    Private Sub btnUpdate_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnUpdate.Click
        Call SetValues()

        Dim cnstrBVs As New OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0; Data Source=U:\Temp\BVs.accdb")
        Dim dsBVVals As New Data.DataSet
        Dim BVadapter As OleDbDataAdapter = New OleDbDataAdapter("Select * FROM L" & intLine, cnstrBVs)

        BVadapter.Fill(dsBVVals, "L" & intLine)

        Dim bvbuilder As OleDbCommandBuilder = New OleDbCommandBuilder(BVadapter)
        BVadapter.InsertCommand = bvbuilder.GetInsertCommand()


        For i = CInt(txtFirstOutput.Text) To CInt(txtLastOutput.Text)
            Me.Text = "Processing file " & i & " of " & CInt(txtLastOutput.Text)
            Dim oInfile As New System.IO.StreamReader("U:\TEMP\" & intLine & "\" & strFarm & intLine & CStr(i) & ".csv")
            Dim aryVarNames() = Split(oInfile.ReadLine, ",")
            Dim tblInfile As New DataTable

            For j = 0 To aryVarNames.Length - 1
                tblInfile.Columns.Add(aryVarNames(j))
            Next j

            Do While Not oInfile.EndOfStream
                Dim aryDataIn() = Split(oInfile.ReadLine, ",")
                tblInfile.Rows.Add(aryDataIn)
            Loop

            For Each Row As DataRow In tblInfile.Rows
                Dim bvnewrow As Data.DataRow = dsBVVals.Tables("L" & intLine).NewRow
                For j = 0 To aryVarNames.Length - 1
                    bvnewrow.Item(aryVarNames(j)) = Row.Item(aryVarNames(j))
                   
                Next
                dsBVVals.Tables("L" & intLine).Rows.Add(bvnewrow)
                bvnewrow.EndEdit()
            Next
            tblInfile = Nothing
        Next

        BVadapter.Update(dsBVVals, "L" & intLine)
        dsBVVals.AcceptChanges()


    End Sub



So, surely there's a way to just slam my vb.net data table into Access without all this damn looping? When I run the Update/Accept changes commands at the end of the process, it takes like 40 minutes to update the DB. Each time I do this, I'm moving around 250k records around. Everything up to the last two lines takes less than a minute, but doing the update is a serious time hog.

I've tried just importing the CSV files directly into excel, but because their formats change so much from file to file, I end up needing to adjust my import steps for every other file and I can't just use a single rule to bring everything in. I tried importing each file into its own table and merging everything together, but access won't let me do it because I apparently wind up with too many fields (which is stupid since the table I'm exporting to using my current method has the same number of fields).

Surely there's a better method than my half-assed hack of a technique?

Viewing all articles
Browse latest Browse all 51036

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>