Matlab Training Session 5: Importing Data
Matlab Training Session 5:
Importing Data
Course Outline
Weeks:
1. Introduction to Matlab and its Interface (Jan 13 2009)
2. Fundamentals (Operators)
3. Fundamentals (Flow)
4. Importing Data
5. Functions and M-Files
6. Plotting (2D and 3D)
7. Statistical Tools in Matlab
8. Analysis and Data Structures
Course Website:
http://www.queensu.ca/neurosci/matlab.php
Week 5 Lecture Outline
Importing Data
A. Week 4 Review
B. Simple Import
C. Handling Files
D. Mini-Project
Functions in Matlab
• In Matlab, each function is a .m file
– It is good protocol to name your .m file the same as
your function name, i.e. funcname.m
• function outargs=funcname(inargs);
input
Function
output
Mini-Project
• Raising any number of numbers to the nth power
• Inputs:
– A vector of numbers to be raised (N1…Nm)
– A vector of powers (P1…Pm)
• Outputs:
– A vector of raised values (N1P1 … NmPm)
– An error flag: 1 if error in calculation, 0 if successful
•
Caveats:
– If only one input is provided, the function should square each
entry, so output = (N12…Nm2) and error flag is 0
– If the length of N and P are not the same, this is an error, return
anything in the output vector and a 1 in the error flag
• Make sure to comment and document the function
Solution
Complex
function [y, e] = raise(x,n)
y = ones(1,length(x));
Simple
function [y, e] = raise(x,n)
if nargin == 1
[y e] = x.^2;
return
elseif nargin == 2
if(length(x)~=length(n))
y = NaN;
e = 1;
return
end
y = x.^n
e = 0;
end
if nargin == 1
[y e] = raise(x,2*ones(1,length(x)));
return
elseif nargin == 2
if(length(x)~=length(n))
y = NaN;
e = 1;
return
end
for(i=1:length(x))
for(j=1:n(i))
y(i) = y(i)*x(i);
end
end
e = 0;
return
end
Importing Data
• Basic issue:
– How do we get data from other sources into
Matlab so that we can play with it?
• Other Issues:
– Where do we get the data?
– What types of data can we import
• Easily or Not
load
• Command opens and imports data from a
standard ASCII file into a matlab variable
• Restrictions
– Data must be constantly sized
– Data must be ASCII
– No other characters
load
• Consider the simple file below
• Create in notepad and save as test1.txt and
text2.txt
• In matlab, set the path to the correct place (ie.
where the file is) and type load(‘test1.txt’)
• Now, type x = load(‘test1.txt’)
test1
12
34
52
48
16 32
load
• Now the file is no longer simple because not
every row has the same amount of characters
• Create in notepad and save as test2.txt and
text2.txt
• type y = load(‘test2.txt’)
– Error!
test2
12
34
5
48
16 32
load
• Now type in the same thing from test1.txt into
Excel and save the workbook as test1.xls
• type y = load(‘test1.xls’)
– What happens?
• Forcing the issue with Excel data
test1
12
34
52
48
16 32
load
• Works for simple and unstructured code
• Powerful and easy to use but limited
• Will likely force you to manually handle
simplifying data which is prone to error
• More complex functions are more flexible
File Handling
• f* functions are associated with file opening,
reading, manipulating, writing, …
• Basic Functions of Interest for opening and
reading generic files in matlab
–
–
–
–
–
fopen
fclose
fseek/ftell/frewind
fscanf
fgetl
fopen
• Opens a file object in matlab that points to the
file of interest
• fid = fopen(‘filepath’)
– absolute directory + filename
• If file of interest is C:\Andrew\Project_1\file.dat
• fid = fopen(‘C:\Andrew\Project_1\file.dat’)
– relative path + filename
• If your matlab path is set to c:\Andrew\Project_1
• fid = fopen(‘file.dat’)
• fid is an integer that represents the file
– Can open multiple files and matlab will assign unique
fids
fclose
• When you are done with a file, it is a good idea
to close it especially if you are opening many
files
• fclose(fid)
What is a File?
• A specific organization of data
• In matlab it is identified with a fid
• Location is specified with a pointer that can be
moved around
fid
Pointer
file_name
Moving the Pointer
• We already know how to assign a fid (fopen)
• To find where the file is pointing:
– x = ftell(fid)
• To point somewhere else
– fseek(fid,offset,origin)
• Move pointer in file fid by offset relative to origin
– Move pointer by a fixed number of bytes
– Origin can be beginning, current, end of file
• To point to the beginning
– frewind(fid)
Getting Data
• Why move the pointer around?
– Get somewhere in the file from where you want data
• [data] = fscanf(fid,format,size)
• Format
– You have to tell matlab the type of data it should be
expecting in the text file so that it can convert it
• ‘%d’, ‘%f’, ‘%c’, ‘%s’
• Size
– You can specify how to organize the imported data
• [m,n] – import the data as m by n, n can be infinite
• Be careful because matlab will mangle your data and not tell
you
Lets Try It
•
Open text1.txt using the fopen command
– Remember to save the fid, we will need it
•
•
Create a variable with the data of text1.txt
Now create another variable y with the data of text1.txt in it by using
fscanf (do not simply copy x)
– What happens here?
– Need to set file pointer to beginning using rewind(fid)
•
•
Now use the size option to import the data with 5 rows and 2
columns
Try the same thing with test2.txt
– It works and fills in the blanks. This is powerful but dangerous
Lets Try It
•
Open text1.txt using the fopen command
– Remember to save the fid, we will need it
fid = fopen('test1.txt)
•
Create a variable with the data of text1.txt
[x] = fscanf(fid,'%f%f')
•
Now create another variable y with the data of text1.txt in it by using
fscanf (do not simply copy x)
[y] = fscanf(fid,'%f%f')
– What happens here?
– Need to set file pointer to beginning using frewind(fid)
Lets Try It
•
Now use the size option to import the data with 5 rows and 2
columns
[data2] = fscanf(fid,'%f%f',[5,2])
- Careful this is the same format as the original data but not the same
organization!!
frewind(fid)
[data3] = fscanf(fid,'%f%f',[2,5])
data3’
- now the data is formatted correctly
• Try the same thing with test2.txt
– It works and fills in the blanks. This is powerful but dangerous
Getting Data
• fgetl returns the next line of the file as a
character array
• You may need to convert these to numbers
>> fid1 = fopen(‘test1.txt’);
>> a_str = fgetl(fid1)
a_str = 1 2
>> a_num = str2num(a_str)
a_num = [1 2]
Realistic File
• A realistic file of data will have header information,
labeled columns and other information embedded within
it.
– See PDXtemp.dat
• Option 1: Manually go through deleting this information
and import using load of fopen commands.
• Option 2: Have matlab delete and format available data
on the fly
Realistic File
• Powerful function textread can be used to
input almost any text file
• Handles (input variables):
– Opening the file
– Ignoring Header Information
– Accepting Column Labels
• Will work for most applications
Realistic File
• Powerful function textread can be used to
input almost any text file
Usage:
[var1 varN] =
textread(‘filename’,’format’,args)
Summary
• Lots of options to load files
– load for basics
– fscanf for complex
– textread for most things
– xlsread for Excel worksheets
• Also saving Excel sheets as tab delimitted
Mini-Project
• Using the textread function, import the full data located in
PDXtemp.dat with the stated names and correct data
types
• Take the resulting temperatures in Fahrenheit and
convert to Celsius
– Make use of Matlab help to learn about and implement the
textread function
– Deg_F = 9/5*Deg_C + 32
Mini-Project
Solution
% Assume that the textfile is saved in the matlab work directory
%
which is automatically in the path
% read data from PDXtemp.txt into 4 variables (assume PDXtemp.txt is in
path)
[month, high_F, low_F, avg_F] = … textread('PDXtemp.txt','%4c%f%f
%f','headerlines', 6);
% convert each temperature variable to celcius
high_C = (5/9).*(high_F - 32);
Low_C = (5/9).*(low_F - 32);
avg_C = (5/9).*(avg_F - 32);
Getting Help
Help and Documentation
Digital
1. Accessible Help from the Matlab Start Menu
2. Updated online help from the Matlab Mathworks website:
http://www.mathworks.com/access/helpdesk/help/techdoc/matlab.html
3. Matlab command prompt function lookup
4. Built in Demo’s
5. Websites
Hard Copy
3. Books, Guides, Reference
The Student Edition of Matlab pub. Mathworks Inc.
Importing Data
Course Outline
Weeks:
1. Introduction to Matlab and its Interface (Jan 13 2009)
2. Fundamentals (Operators)
3. Fundamentals (Flow)
4. Importing Data
5. Functions and M-Files
6. Plotting (2D and 3D)
7. Statistical Tools in Matlab
8. Analysis and Data Structures
Course Website:
http://www.queensu.ca/neurosci/matlab.php
Week 5 Lecture Outline
Importing Data
A. Week 4 Review
B. Simple Import
C. Handling Files
D. Mini-Project
Functions in Matlab
• In Matlab, each function is a .m file
– It is good protocol to name your .m file the same as
your function name, i.e. funcname.m
• function outargs=funcname(inargs);
input
Function
output
Mini-Project
• Raising any number of numbers to the nth power
• Inputs:
– A vector of numbers to be raised (N1…Nm)
– A vector of powers (P1…Pm)
• Outputs:
– A vector of raised values (N1P1 … NmPm)
– An error flag: 1 if error in calculation, 0 if successful
•
Caveats:
– If only one input is provided, the function should square each
entry, so output = (N12…Nm2) and error flag is 0
– If the length of N and P are not the same, this is an error, return
anything in the output vector and a 1 in the error flag
• Make sure to comment and document the function
Solution
Complex
function [y, e] = raise(x,n)
y = ones(1,length(x));
Simple
function [y, e] = raise(x,n)
if nargin == 1
[y e] = x.^2;
return
elseif nargin == 2
if(length(x)~=length(n))
y = NaN;
e = 1;
return
end
y = x.^n
e = 0;
end
if nargin == 1
[y e] = raise(x,2*ones(1,length(x)));
return
elseif nargin == 2
if(length(x)~=length(n))
y = NaN;
e = 1;
return
end
for(i=1:length(x))
for(j=1:n(i))
y(i) = y(i)*x(i);
end
end
e = 0;
return
end
Importing Data
• Basic issue:
– How do we get data from other sources into
Matlab so that we can play with it?
• Other Issues:
– Where do we get the data?
– What types of data can we import
• Easily or Not
load
• Command opens and imports data from a
standard ASCII file into a matlab variable
• Restrictions
– Data must be constantly sized
– Data must be ASCII
– No other characters
load
• Consider the simple file below
• Create in notepad and save as test1.txt and
text2.txt
• In matlab, set the path to the correct place (ie.
where the file is) and type load(‘test1.txt’)
• Now, type x = load(‘test1.txt’)
test1
12
34
52
48
16 32
load
• Now the file is no longer simple because not
every row has the same amount of characters
• Create in notepad and save as test2.txt and
text2.txt
• type y = load(‘test2.txt’)
– Error!
test2
12
34
5
48
16 32
load
• Now type in the same thing from test1.txt into
Excel and save the workbook as test1.xls
• type y = load(‘test1.xls’)
– What happens?
• Forcing the issue with Excel data
test1
12
34
52
48
16 32
load
• Works for simple and unstructured code
• Powerful and easy to use but limited
• Will likely force you to manually handle
simplifying data which is prone to error
• More complex functions are more flexible
File Handling
• f* functions are associated with file opening,
reading, manipulating, writing, …
• Basic Functions of Interest for opening and
reading generic files in matlab
–
–
–
–
–
fopen
fclose
fseek/ftell/frewind
fscanf
fgetl
fopen
• Opens a file object in matlab that points to the
file of interest
• fid = fopen(‘filepath’)
– absolute directory + filename
• If file of interest is C:\Andrew\Project_1\file.dat
• fid = fopen(‘C:\Andrew\Project_1\file.dat’)
– relative path + filename
• If your matlab path is set to c:\Andrew\Project_1
• fid = fopen(‘file.dat’)
• fid is an integer that represents the file
– Can open multiple files and matlab will assign unique
fids
fclose
• When you are done with a file, it is a good idea
to close it especially if you are opening many
files
• fclose(fid)
What is a File?
• A specific organization of data
• In matlab it is identified with a fid
• Location is specified with a pointer that can be
moved around
fid
Pointer
file_name
Moving the Pointer
• We already know how to assign a fid (fopen)
• To find where the file is pointing:
– x = ftell(fid)
• To point somewhere else
– fseek(fid,offset,origin)
• Move pointer in file fid by offset relative to origin
– Move pointer by a fixed number of bytes
– Origin can be beginning, current, end of file
• To point to the beginning
– frewind(fid)
Getting Data
• Why move the pointer around?
– Get somewhere in the file from where you want data
• [data] = fscanf(fid,format,size)
• Format
– You have to tell matlab the type of data it should be
expecting in the text file so that it can convert it
• ‘%d’, ‘%f’, ‘%c’, ‘%s’
• Size
– You can specify how to organize the imported data
• [m,n] – import the data as m by n, n can be infinite
• Be careful because matlab will mangle your data and not tell
you
Lets Try It
•
Open text1.txt using the fopen command
– Remember to save the fid, we will need it
•
•
Create a variable with the data of text1.txt
Now create another variable y with the data of text1.txt in it by using
fscanf (do not simply copy x)
– What happens here?
– Need to set file pointer to beginning using rewind(fid)
•
•
Now use the size option to import the data with 5 rows and 2
columns
Try the same thing with test2.txt
– It works and fills in the blanks. This is powerful but dangerous
Lets Try It
•
Open text1.txt using the fopen command
– Remember to save the fid, we will need it
fid = fopen('test1.txt)
•
Create a variable with the data of text1.txt
[x] = fscanf(fid,'%f%f')
•
Now create another variable y with the data of text1.txt in it by using
fscanf (do not simply copy x)
[y] = fscanf(fid,'%f%f')
– What happens here?
– Need to set file pointer to beginning using frewind(fid)
Lets Try It
•
Now use the size option to import the data with 5 rows and 2
columns
[data2] = fscanf(fid,'%f%f',[5,2])
- Careful this is the same format as the original data but not the same
organization!!
frewind(fid)
[data3] = fscanf(fid,'%f%f',[2,5])
data3’
- now the data is formatted correctly
• Try the same thing with test2.txt
– It works and fills in the blanks. This is powerful but dangerous
Getting Data
• fgetl returns the next line of the file as a
character array
• You may need to convert these to numbers
>> fid1 = fopen(‘test1.txt’);
>> a_str = fgetl(fid1)
a_str = 1 2
>> a_num = str2num(a_str)
a_num = [1 2]
Realistic File
• A realistic file of data will have header information,
labeled columns and other information embedded within
it.
– See PDXtemp.dat
• Option 1: Manually go through deleting this information
and import using load of fopen commands.
• Option 2: Have matlab delete and format available data
on the fly
Realistic File
• Powerful function textread can be used to
input almost any text file
• Handles (input variables):
– Opening the file
– Ignoring Header Information
– Accepting Column Labels
• Will work for most applications
Realistic File
• Powerful function textread can be used to
input almost any text file
Usage:
[var1 varN] =
textread(‘filename’,’format’,args)
Summary
• Lots of options to load files
– load for basics
– fscanf for complex
– textread for most things
– xlsread for Excel worksheets
• Also saving Excel sheets as tab delimitted
Mini-Project
• Using the textread function, import the full data located in
PDXtemp.dat with the stated names and correct data
types
• Take the resulting temperatures in Fahrenheit and
convert to Celsius
– Make use of Matlab help to learn about and implement the
textread function
– Deg_F = 9/5*Deg_C + 32
Mini-Project
Solution
% Assume that the textfile is saved in the matlab work directory
%
which is automatically in the path
% read data from PDXtemp.txt into 4 variables (assume PDXtemp.txt is in
path)
[month, high_F, low_F, avg_F] = … textread('PDXtemp.txt','%4c%f%f
%f','headerlines', 6);
% convert each temperature variable to celcius
high_C = (5/9).*(high_F - 32);
Low_C = (5/9).*(low_F - 32);
avg_C = (5/9).*(avg_F - 32);
Getting Help
Help and Documentation
Digital
1. Accessible Help from the Matlab Start Menu
2. Updated online help from the Matlab Mathworks website:
http://www.mathworks.com/access/helpdesk/help/techdoc/matlab.html
3. Matlab command prompt function lookup
4. Built in Demo’s
5. Websites
Hard Copy
3. Books, Guides, Reference
The Student Edition of Matlab pub. Mathworks Inc.