Problem Solution Discussion Getting LOAD DATA to Cough Up More Information

10.8.2 Solution

Tell LOAD DATA t o ignore t he new records, or t o replace t he old ones.

10.8.3 Discussion

By default , an error occurs if you at t em pt t o load a record t hat duplicat es an exist ing record in t he colum n or colum ns t hat form a PRIMARY KEY or UNIQUE index. To cont rol t his behavior, specify IGNORE or REPLACE aft er t he filenam e t o t ell MySQL t o eit her ignore duplicat e records or t o replace old records wit h t he new ones. Suppose you periodically receive m et eorological dat a about current weat her condit ions from various m onit oring st at ions, and t hat you st ore m easurem ent s of various t ypes from t hese st at ions in a t able t hat looks like t his: CREATE TABLE weatherdata station INT UNSIGNED NOT NULL, type ENUMprecip,temp,cloudiness,humidity,barometer NOT NULL, value FLOAT, UNIQUE station, type ; To m ake sure t hat you have only one record for each st at ion for each t ype of m easurem ent , t he t able includes a unique key on t he com binat ion of st at ion I D and m easurem ent t ype. The t able is int ended t o hold only current condit ions, so when new m easurem ent s for a given st at ion are loaded int o t he t able, t hey should kick out t he st at ions previous m easurem ent s. To accom plish t his, use t he REPLACE keyword: mysql LOAD DATA LOCAL INFILE data.txt REPLACE INTO TABLE weatherdata;

10.9 Getting LOAD DATA to Cough Up More Information

10.9.1 Problem

LOAD DATA doesnt t ell you m uch about problem s in t he dat afile.

10.9.2 Solution

There is no solut ion. Well, m aybe t here is.

10.9.3 Discussion

When a LOAD DATA st at em ent finishes, it ret urns a line of inform at ion t hat t ells you how m any errors or dat a conversion problem s occurred. Suppose you load a file int o a t able and see t he following m essage when LOAD DATA finishes. Records: 134 Deleted: 0 Skipped: 2 Warnings: 13 These values provide som e general inform at ion about t he im port operat ion: • Records indicat es t he num ber of records found in t he file. • Deleted and Skipped are relat ed t o t reat m ent of input records t hat duplicat e exist ing t able records on unique index values. Deleted indicat es how m any records were delet ed from t he t able and replaced by input records, and Skipped indicat es how m any input records were ignored in favor of exist ing records. • Warnings is som et hing of a cat ch- all t hat indicat es t he num ber of problem s found while loading dat a values int o colum ns. Eit her a value st ores int o a colum n properly, or it doesnt . I n t he lat t er case, t he value ends up in MySQL as som et hing different and MySQL count s it as a warning. St oring a st ring abc int o a num eric colum n result s in a st ored value of , for exam ple. What do t hese values t ell you? The Records value norm ally should m at ch t he num ber of lines in t he input file. I f it is different t han t he files line count , t hat s a sign t hat MySQL is int erpret ing t he file as having a form at t hat differs from t he form at it act ually has. I n t his case, youre likely also t o see a high Warnings value, which indicat es t hat m any values had t o be convert ed because t hey didnt m at ch t he expect ed dat a t ype. The solut ion t o t his problem oft en is t o specify t he pr oper FIELDS and LINES clauses. Ot herwise, t he values m ay not t ell you a lot . You cant t ell from t hese num bers which input records had problem s or which colum ns were bad. There is som e work being done for MySQL 4 t o m ake addit ional warning inform at ion available. I n t he m eant im e, see Recipe 10.38 for a script t hat exam ines your dat afile and at t em pt s t o pinpoint t roublesom e dat a values.

10.10 Dont Assume LOAD DATA Knows More than It Does