1

so this is a relatively simple question. I have a postgresql table which I need to dump to an xml file on my local machine. I've read all the documentation at https://www.postgresql.org/docs/current/static/functions-xml.html and https://www.postgresql.org/docs/9.4/static/sql-copy.html and I have come up with the following command through the psql command line:

COPY (SELECT table_to_xml(SELECT FROM Database public.'table', true, false, '')) to 'C:/users/me/file.xml';

However I keep getting a syntax error at or near the "FROM". I have searched for the past hour through every SO link that looked useful (Postgresql tables exists, but getting "relation does not exist" when querying | Cannot simply use PostgreSQL table name ("relation does not exist") | using copy in postgresql? | Postgres Data to XML) and every YouTube video that sounded remotely like what I needed, but to no avail. I've tried with and without the FROM statement, tried with a SELECT * in front of the FROM, etc... no luck. There must be something terribly simple I'm missing. Any ideas?

6
  • Can you provide the output for : SELECT * FROM information_schema.tables where table_name = 'your_table_name' Commented May 10, 2018 at 16:23
  • Curious, does table_to_xml() query work before even copying to text file? Commented May 10, 2018 at 16:25
  • @Vivek When I execute the query SELECT * FROM information_schema.tables where table_name = 'your_table_name' on my PGAdmin 4 I get the following results: A table with quite a few headers that has one row. The headers are table_catalog (Database Name), table_schema (public), table_name (Name of My Table), table_type (BASE TABLE), etc... the rest are null Commented May 10, 2018 at 16:43
  • Instead of giving public as database name put the value that’s returns as database, it’s all are try as I don’t have pg session right now Commented May 10, 2018 at 16:49
  • @Vivek So good news and bad news... Good news is I tried as you suggested to execute the command COPY (SELECT table_to_xml(Database.'table', true, false, '')) to 'C:/users/me/file.xml'; And it did not give me an error... however no file was created, and by the looks of my Task manager nothing else is happening. Commented May 10, 2018 at 17:16

3 Answers 3

0

Try this, it seems the syntax is wrong, which you tried.

COPY (SELECT table_to_xml('table', true, false, '')) to 'C:/users/me/file.xml';

Alternative, you can try:

psql -p5432 db_name \copy (SELECT table_to_xml('table', true, false, '')) TO 'C:/users/me/file.xml';
Sign up to request clarification or add additional context in comments.

4 Comments

Thank you for the quick answer, but I have already tried this. I get the error that that relation does not exist when it does. That is why I added the Database public.'table' portion to the original query. If I try COPY (SELECT table_to_xml(Database public.'table', true, false, '')) to 'C:\users\me\file.xml'; I get a syntax error at or near 'public'
can you use 'public.table' instead
I have also added one more way to do the same, using "psql" command this also works.
I have attempted both of your suggestions to no avail. I am still getting the error that the relation does not exist. I have triple checked that it is spelled correctly and typed exactly in the case I created it in... I don't understand how it still cannot find the relation. Is there perhaps some setting in the PGAdmin tool I could set to make it recognize the relation?
0

OK so update. After spending the majority of my day struggling with this I have come up with a partial solution. Using information from here and Permission denied when trying to import a CSV file from PGAdmin I was able to get a command to work in PGAdmin 4 sort of...

COPY (SELECT table_to_xml('table_name', true, false, '')) to 'C:\XML\Table.xml';

The above command when run in PGAdmin will apparently do something I only assume is what I want. I have not had time to check if it actually creates a working XML file of the table if I use a smaller dataset. Yet if I follow the instructions in the before mentioned link and create a folder with a blank XML file in it and set the permissions to allow everyone to read/write to the folder the command executes. However it uses an obscene amount of memory. To write a table which is only around 2GB in size to XML the program consumed all 36GB of my memory before crashing itself. I don't have time to continue testing this, because I found that I could get to where I needed to be using a dump to a text file. anyway I hope this helps someone.

Comments

0

The copy-statement doesn't work for me - runs out of RAM.
Exporting with C# works fine, though (this still loads all data into memory):

public static void DataToXML()
{
    Npgsql.NpgsqlConnectionStringBuilder csb = new Npgsql.NpgsqlConnectionStringBuilder();
    csb.Database = "YourDB";
    csb.Host = "localhost";
    csb.Port = 5432;
    csb.IntegratedSecurity = true;
    csb.Username = "postgres";
    // csb.Password = "";


    using (System.Data.DataTable dt = new System.Data.DataTable())
    {

        dt.TableName = "record";

        using (System.Data.DataSet ds = new System.Data.DataSet("geoip_blocks_temp"))
        {
            ds.Tables.Add(dt);
            // dt.Namespace = "foo";

            using (System.Data.Common.DbConnection con = Npgsql.NpgsqlFactory.Instance.CreateConnection())
            {
                con.ConnectionString = csb.ConnectionString;

                using (System.Data.Common.DbCommand cmd = con.CreateCommand())
                {
                    cmd.CommandText = "SELECT * FROM geoip.geoip_blocks_temp";

                    using (System.Data.Common.DbDataAdapter da = Npgsql.NpgsqlFactory.Instance.CreateDataAdapter())
                    {
                        da.SelectCommand = cmd;

                        if (con.State != System.Data.ConnectionState.Open)
                            con.Open();

                        da.Fill(dt);

                        if (con.State != System.Data.ConnectionState.Open)
                            con.Close();
                    } // End Using da 

                } // End Using cmd 

            } // End Using con 


            //using (System.IO.Stream fs = System.IO.File.OpenWrite(@"D:\geoip_blocks_temp.xml"))
            //{
            //    using (System.IO.TextWriter sw = new System.IO.StreamWriter(fs, System.Text.Encoding.UTF8))
            //    {
            //        // System.IO.StringWriter sw = new System.IO.StringWriter();
            //        // dt.WriteXml(sw, System.Data.XmlWriteMode.IgnoreSchema);
            //        dt.WriteXml(sw, System.Data.XmlWriteMode.IgnoreSchema);
            //    } // End Using sw 

            //} // End Using fs 


            System.Xml.XmlWriterSettings xs = new System.Xml.XmlWriterSettings();
            xs.Indent = true;
            xs.IndentChars = "    ";
            xs.NewLineChars = System.Environment.NewLine;
            xs.OmitXmlDeclaration = false;
            // xs.Encoding = System.Text.Encoding.UTF8; // doesn't work with pgsql 
            xs.Encoding = new System.Text.UTF8Encoding(false);

            // <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
            using (System.Xml.XmlWriter writer = System.Xml.XmlWriter.Create(@"D:\geoip_blocks_temp.xml", xs))
            {
                dt.WriteXml(writer, System.Data.XmlWriteMode.IgnoreSchema);
            }

            System.Console.WriteLine(dt.Rows.Count);
        } // End Using ds 

    } // End Using dt 

} // End Sub DataToXML 

Or more memory-conserving:

public static void LargeDataToXML()
{
    string table_schema = "geoip";
    string table_name = "geoip_blocks_temp";

    // table_schema = "public";
    // table_name = "t_sys_language_monthnames";


    System.Xml.XmlWriterSettings xs = new System.Xml.XmlWriterSettings();
    xs.Indent = true;
    xs.IndentChars = "    ";
    xs.NewLineChars = System.Environment.NewLine;
    xs.OmitXmlDeclaration = false; // // <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    // xs.Encoding = System.Text.Encoding.UTF8; // doesn't work with pgsql 
    xs.Encoding = new System.Text.UTF8Encoding(false);

    string exportFilename = System.IO.Path.Combine(@"d:\", table_name + ".xml");

    using (System.Xml.XmlWriter writer = System.Xml.XmlWriter.Create(exportFilename, xs))
    {
        writer.WriteStartDocument();
        writer.WriteStartElement(table_name);

        writer.WriteAttributeString("xmlns", "xsi", null, System.Xml.Schema.XmlSchema.InstanceNamespace);
        // writer.WriteAttributeString("xsi", "schemaLocation", null, System.Xml.Schema.XmlSchema.InstanceNamespace);



        using (System.Data.Common.DbConnection con = Npgsql.NpgsqlFactory.Instance.CreateConnection())
        {
            con.ConnectionString = GetCS();

            using (System.Data.Common.DbCommand cmd = con.CreateCommand())
            {
                cmd.CommandText = "SELECT * FROM " + table_schema + "." + table_name;


                if (con.State != System.Data.ConnectionState.Open)
                    con.Open();

                using (System.Data.Common.DbDataReader dr = cmd.ExecuteReader(System.Data.CommandBehavior.SequentialAccess))
                {

                    if (dr.HasRows)
                    {
                        int fc = dr.FieldCount;

                        string[] columnNames = new string[fc];
                        // System.Type[] columnTypes = new System.Type[fc];

                        for (int i = 0; i < dr.FieldCount; ++i)
                        {
                            columnNames[i] = dr.GetName(i);
                            // columnTypes[i] = dr.GetFieldType(i);
                        } // Next i 

                        while (dr.Read())
                        {
                            // object[] thisRow = new object[dr.FieldCount];

                            writer.WriteStartElement("record");

                            for (int i = 0; i < fc; ++i)
                            {
                                writer.WriteStartElement(columnNames[i]);
                                object obj = dr.GetValue(i);

                                if (obj != System.DBNull.Value)
                                {
                                    writer.WriteValue(obj);
                                }
                                else
                                    writer.WriteAttributeString("xsi", "nil", System.Xml.Schema.XmlSchema.InstanceNamespace, "true");

                                writer.WriteEndElement();
                            } // Next i

                            writer.WriteEndElement();

                        } // Whend 

                    } // End if (dr.HasRows) 

                } // End Using dr 

                if (con.State != System.Data.ConnectionState.Open)
                    con.Close();
            } // End Using cmd 

        } // End Using con 

        writer.WriteEndElement();
    } // ENd Using writer 

} // End Sub LargeDataToXML 

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.