To copy m ult iple t ables over t he net work, nam e t hem all following t he dat abase argum ent of t he m ysqldum p com m and. To copy an ent ire dat abase, dont specify any t able nam es aft er t he
dat abase nam e. m ysqldum p will dum p all t he t ables cont ained in t he dat abase. I f youre t hinking about invoking m ysqldum p wit h t he - - all- dat abases opt ion t o send all your
dat abases t o anot her server, consider t hat t he out put will include t he t ables in t he
mysql
dat abase t hat cont ains t he grant t ables. I f t he rem ot e server has a different user populat ion, you probably dont want t o replace t hat servers grant t ables
10.18 Writing Your Own Export Programs
10.18.1 Problem
MySQLs built - in export capabilit ies dont suffice.
10.18.2 Solution
Writ e your own ut ilit ies.
10.18.3 Discussion
When exist ing soft ware doesnt do what you want , you can writ e your own program s t o export dat a. This sect ion show s how t o writ e a Perl script , m ysql_t o_t ext .pl, t hat execut es an
arbit rary query and export s it in t he form at you specify. I t writ es out put t o t he client host and can include a row of colum n labels neit her of which
SELECT
...
INTO OUTFILE
can do . I t produces m ult iple out put form at s m ore easily t han by using m ysql wit h a post processor, and it
writ es t o t he client host , unlike m ysqldum p, which can writ e only SQL- form at out put t o t he client .
m ysql_t o_t ext .pl is based on t he Text : : CSV_XS m odule, which youll need t o obt ain if it s not inst alled on your syst em . Once it s inst alled, you can read t he docum ent at ion like so:
perldoc Text::CSV_XS
This m odule is convenient because all you have t o do is provide an array of colum n values, and it will package t hem up int o a properly form at t ed out put line. This m akes it relat ively
t rivial t o convert query out put t o CSV form at . But t he real benefit of using t he Text : : CSV_XS m odule is t hat it s configurable; you can t ell it what kind of delim it er and quot e charact ers t o
use. This m eans t hat alt hough t he m odule produces CSV form at by default , you can configure it t o writ e a variet y of out put form at s. For exam ple, if you set t he delim it er t o t ab and t he
quot e charact er t o
undef,
Text : : CSV_XS generat es t ab-delim it ed out put . Well t ake advant age of t hat flexibilit y in t his sect ion for writ ing m ysql_t o_t ext .pl, and lat er in
Recipe 10.19
t o writ e a file-processing ut ilit y t hat convert s files from one form at t o anot her. m ysql_t o_t ext .pl accept s several com m and- line opt ions. Som e of t hese are for specifying
MySQL connect ion param et ers such as - - user, - - password, and - - host . Youre already
fam iliar wit h t hese, because t heyre used by t he st andard MySQL client s like m ysql. The script also can obt ain connect ion param et ers from an opt ion file, if you specify a
[client]
group in t he file. The ot her opt ions t hat m ysql_t o_t ext .pl accept s are as follow s:
- - execut e
= query
, - e
query
Execut e query
and ex por t it s out put .
- - t able
= tbl_name
, - t
tbl_name
Export t he cont ent s of t he nam ed t able. This is equivalent t o using - - ex ecut e t o specify a query
v alue of SELECT
FROM tbl_name
.
- - labels
Writ e an init ial row of colum n labels.
- - delim
= str
Set t he colum n delim it er sequence t o str
. The opt ion value m ay consist of one or m or e char act er s. The default is t o use t abs.
- - quot e
= c
Set t he colum n value quot e char act er t o c
. The default is t o not quot e anyt hing.
- - eol
= str
Set t he end- of- line sequence t o str
. The opt ion value m ay consist of one or m ore charact ers. The default is t o use linefeeds.
The default s for t he - - delim , - - quot e, and - - eol opt ions correspond t o t hose used by
LOAD DATA
and
SELECT
...
INTO OUTFILE
. The final argum ent on t he com m and line should be t he dat abase nam e, unless it s im plicit in
t he query. For exam ple, t hese t wo com m ands are equivalent ; each export s t he
passwd
t able in colon- delim it ed form at int o a file nam ed t m p:
mysql_to_text.pl --delim=: --table=passwd cookbook tmp mysql_to_text.pl --delim=: --table=cookbook.passwd tmp
To generat e CSV out put wit h CRLF line t erm inat ors inst ead, use a com m and like t his:
mysql_to_text.pl --delim=, --quote=\ --eol=\r\n \ --table=cookbook.passwd tmp
That s a general descript ion of how you use m ysql_t o_t ext .pl. Now let s discuss how it works. The init ial part of t he m ysql_t o_t ext .pl script declares a few variables, t hen processes t he
com m and- line argum ent s, using opt ion- processing t echniques developed in Recipe 2.11
. As it happens, m ost of t he code in t he script act ually is devot ed t o processing t he com m and- line
argum ent s and get t ing set up t o run t he query usrbinperl -w
mysql_to_text.pl - export MySQL query output in user-specified text format
Usage: mysql_to_text.pl [ options ] [db_name] text_file use strict;
use DBI; use Text::CSV_XS;
use Getopt::Long; Getopt::Long::ignorecase = 0; options are case sensitive
Getopt::Long::bundling = 1; allow short options to be bundled my prog = mysql_to_text.pl;
... construct usage message variable usage not shown ...
Variables for command line options - all undefined initially except for output structure, which is set to be tab-delimited,
linefeed-terminated. my help;
my host_name, password, port_num, socket_name, user_name, db_name; my query, tbl_name;
my labels; my delim = \t;
my quote; my eol = \n;
GetOptions =i means an integer argument is required after the option
=s means a string argument is required after the option :s means a string argument is optional after the option
help = \help, print help message host|h=s = \host_name, server host
password|p:s = \password, password port|P=i = \port_num, port number
socket|S=s = \socket_name, socket name user|u=s = \user_name, username
execute|e=s = \query, query to execute table|t=s = \tbl_name, table to export
labels|l = \labels, generate row of column labels delim=s = \delim, column delimiter
quote=s = \quote, column quoting character eol=s = \eol end-of-line record delimiter
or die usage\n; die usage\n if defined help;
db_name = shift ARGV if ARGV; One of --execute or --table must be specified, but not both
die You must specify a query or a table name\n\nusage\n if defined query defined tbl_name;
die You cannot specify both a query and a table name\n\nusage\n
if defined query defined tbl_name; If table name was given, convert it to a query that selects entire table
query = SELECT FROM tbl_name if defined tbl_name; convert definedundefined state into truefalse
labels = defined labels; interpret special chars in the file structure options
quote = interpret_option quote; delim = interpret_option delim;
eol = interpret_option eol;
The
interpret_option
funct ion processes escape and hex sequences for t he - - delim , - - quot e, and - - eol opt ions.
\n
,
\r
,
\t
, and
\0
are int erpret ed as linefeed, carriage ret urn, t ab, and t he ASCI I NUL charact er. Hex values m ay be given in t he form
0x nn
for exam ple,
0x0d
indicat es a carriage ret urn . The funct ion is not shown here; you can exam ine t he script source t o see how it w orks.
Aft er processing t he com m and- line opt ions, t he script const ruct s t he dat a source nam e DSN and connect s t o t he server:
my dsn = DBI:mysql:; dsn .= ;database=db_name if db_name;
dsn .= ;host=host_name if host_name; dsn .= ;port=port_num if port_num;
dsn .= ;mysql_socket=socket_name if socket_name; read [client] group parameters from standard option files
dsn .= ;mysql_read_default_group=client; my dbh = DBI-connect dsn, user_name, password,
{PrintError = 0, RaiseError = 1};
The dat abase nam e com es from t he com m and line. Connect ion param et ers can com e from t he com m and line or an opt ion file. Use of MySQL opt ion files is covered in
Recipe 2.11 .
Aft er est ablishing a connect ion t o MySQL, t he script is ready t o execut e t he query and produce som e out put . This is where t he Text : : CSV_XS m odule com es int o play. First , we creat e a CSV
obj ect by calling
new
, which t akes an opt ional hash of opt ions t hat cont rol how t he obj ect handles dat a lines. Then t he script prepares and execut es t he query, print s a row of colum n
labels if t he user specified t he - - labels opt ion , and writ es t he rows of t he result set : my csv = Text::CSV_XS-new {
sep_char = delim, quote_char = quote,
escape_char = quote, eol = eol,
binary = 1 };
my sth = dbh-prepare query; sth-execute ;
if labels write row of column labels
{ csv-combine {sth-{NAME}} or die cannot process column
labels\n; print csv-string ;
} my count = 0;
while my val = sth-fetchrow_array {
++count; csv-combine val or die cannot process column values, row
count\n; print csv-string ;
}
The
sep_char
and
quote_char
opt ions in t he
name
call set t he colum n delim it er sequence and quot ing charact er. The
escape_char
opt ion is set t o t he sam e value as
quote_char
so t hat inst ances of t he quot e charact er occurring wit hin dat a values will be doubled in t he out put . The
eol
opt ion indicat es t he line- t erm inat ion sequence. Norm ally, Text : : CSV_XS leaves it t o you t o print t he t erm inat or for out put lines. By passing a non-
undef eol
value t o
new
, t he m odule adds t hat value t o every out put line aut om at ically. The
binary
opt ion is useful for processing dat a values t hat cont ain binary charact ers. The colum n labels are available in
sth-{NAME}
aft er invoking
execute
. Each line of out put is produced using
combine
and
string
. The
combine
m et hod t akes an array of values and convert s t hem t o a properly form at t ed st ring.
string
ret urns t he st ring so w e can print it .
10.19 Converting Datafiles from One Format to Another
