November 7, 2009

Administration
Faculty
Staff
Students
Jobs

Projects
Seminars
Working Papers
Publications
Dissertations

Prospective  Students

Programs
Courses
Course Schedule

Data Archive
Library
Pop Index
NIH Public
    Access Policy

Calendar

CRCW
CHW
CMD
PUM
OPR Mail

Search

Reading Data Into Stata

This lesson will cover in more detail how to get data into Stata. By the end of the lesson you should be able to read a raw data file into Stata using the "infile" and "infix" commands. We will also briefly talk about the "insheet" command for reading data from a spreadsheet program such as Excel.

Often, the data we want to analyze is not already in Stata (.dta) format. What we will work on now is how to get data that are not in Stata format into Stata.

Before we get into the details, let's just note that sometimes data come in SAS or SPSS format, or with SAS or SPSS programs to read the data. In these instances, it is much easier to use a program like StatTransfer or DBMS/Copy to convert the data from one of these formats to Stata.

You may even have data in an Excel or other spreadsheet program. You can use StatTransfer or DBMS/Copy for these files as well, or you can save the spreadsheet as a tab-delimited file and use the "insheet" command to read it into Stata. This is pretty easy to do, so you can try it on your own with the instructions from our refresher lesson

If you do have to write your own program to read data into Stata, then you have two basic choices: "infile" and "infix". The infile command has more capabilities than infix, but is also more complex. Essentially, the choice comes down to this: if your codebook has "start" and "length" information for your variables, or your variables are separated by spaces (not tabs or commas) then use infile. If your codebook has "start" and "end" column information, then use infix.

Some Terminology

• Codebook: A document that describes in detail the data with which you are working. Although there is no standard format for a codebook, a good one has the full wording of the questions and answers, a list of all the codes or values used to enter the data, and either the begin and end columns or the begin column and the length of the variable.
• Dictionary: There are two types of dictionaries: a data dictionary and a Stata dictionary. A data dictionary is a document that lists all the variables and their locations (columns) in the data file, and, sometimes, the values for those variables. These are appropriate for any statistical package. A Stata dictionary is a program or file that Stata uses to read a raw data file - the information in this program can be obtained from either a data dictionary or a codebook. For our purposes, "dictionary" will refer only to a Stata dictionary.
• Observation: An observation is a unit of analysis - a respondent in a survey, for example.
• Record: A record is one line in a raw data file. Sometimes there is more than one record per observation or there are different types of records in a single file. Records are sometimes referred to as "cards."
• Column: a column actually refers to a single character, including spaces, in a raw data file. This is not the same as a column in a spreadsheet.
• Length: The number of columns (or characters) a variable in a raw data file will occupy. Sometimes this also means the number of bytes the same variable will take in a system file.
• Raw data: Raw data are data that have not been read into Stata (or some other package). If you were to open this file you would see numbers and, perhaps, letters. You need a codebook or data dictionary to be able to read the data into Stata. These are sometimes called "text" or "ascii" files.
• Flat: a flat file is a raw data file that has one record for each observation. These are sometimes called "rectangular" files.
• Multiple Records: Sometimes there is more than one record or line of data for each observation in a raw data file. Each record has a different set of variables on it, so each record must be read differently.
• Hierarchical: Sometimes a raw data file will have different types or levels of information on different records. An example is when a survey collects information about a household, each family within that household, and each person within each family. The different types of variables will be on different records within the raw data file.

• String variable: a string variable is one that has letters and/or numbers as opposed to just numbers. An example would be a person's name. Numbers can be treated as strings, but strings cannot be treated as numbers. Strings are also referred to as "character" or "alphanumeric" variables.
• Numeric variable: These are numbers, plain and simple. Decimals, commas, and minus signs are the only acceptable non-number characters allowed.
• Variable label: A variable label is a short description of a variable. These appear in the Variable window in Stata and also in the output. They are not required, but make the output easier to understand.
• Value label: Like variable labels, value labels make the output easier to read.  Instead of printing “1”, “2”, “3”, Stata will print “Yes”, “No”, “Maybe”.

Infile

This is a sample dictionary using the "infile" command. These statements are entered into a separate file called "demo.dct".

infile dictionary {
    _column(1)           month     %2f   "Calendar Month"
    _column(3)           year      %4f   "Calendar Year"
    _column(7)           employed  %1f   "Employment status that month"
    _column(8)           mugged    %1f   "Perpetrator/victim/witness of assualt"
    _column(9)           gender    %1f   "Gender"
   _column(10)           income    %5f   "That month's income"
   _column(15)    str6   d_income  %6s   "Income with a dollar sign"
   _column(21)    str13  str_num   %13s  "Number read in as a string"
   _column(34)           hhid      %3f   "Household ID"
   _column(37)           famid     %1f   "Family ID"
   _column(38)           perid     %1f   "Person ID"
   _column(39)           citynum   %1f   "City ID"
   _column(40)    str15  cityname  %15s  "City name"
   _column(55)           b_day     %2f   "Day of R's Birth"
   _column(57)           b_month   %2f   "Month of R's birth"
   _column(59)           b_year    %4f   "Year of R's birth"
   _column(63)           i_day     %2f   "Interview day"
   _column(65)           i_month   %2f   "Interview month"
   _column(67)    str10  str_date  %10s  "String Date"
   _column(77)           rand      %21f  "Just a random number"
}

Infix

Here is a sample infix dictionary. Like the infile dictionary, it is entered into a separate file called "demo.dct".

infix dictionary{
      month 1-2 
      year  3-6
      employed 7
      mugged 8
      gender 9
      income 10-14
str6  d_income 15-20
str13 str_num 21-33
      hhid 34-36
      famid 37
      perid 38
      citynum 39
str15 cityname 40-54
      b_day 55-56
      b_month 57-58
      b_year 59-62
      i_day 63-64
      i_month 65-66
str10 str_date 67-76
      rand 77-96
}

Saving Your Data

Of course, once you’ve read your data into Stata, you’ll want to save it.

Documenting Your Data

Once you have your data loaded into Stata, you'll want to document it so you know what the variables are and how they are coded. You can do this with variable and value labels, as well as having good records of the data using Stata's "codebook" and "describe" commands.

In this lesson you will learn how to create and modify variable and value labels, create notes for your data and variables as well as make good use of the codebook and describe commands.

Variable and Value Labels

The output from the describe and codebook commands can be very useful, but it can be even more useful if you have defined variable and value labels. Variable and value labels also make the output from other commands easier to read as well. A variable label is a short description of the variable itself and what it is intended to represent. Value labels are a short description of the individual values a variable may have. Used together, they document your program as well as your output.

Here are a few rules and tips for labels:

• Variable labels can be up to 80 characters long in Intercooled Stata, 244 in Special Edition, anything beyond that gets truncated.
• Often, a variable label is simply the text of a question in a survey. If the question is more than 80 characters, you must re-word the question be sure that you do not change the meaning of the question by doing so!!! This may seem obvious, but it has happened.
• If you do not use question numbers as your variable names, then include the question number in the variable label.
• Value labels have the same limits as variable labels, but since many procedures only print the first 12 characters, you would be wise to keep them rather short. Again, be sure not to change the meaning.
• Only integers and special missing values can be labeled.
• One label can be applied to many variables; if you have several yes/no questions, you only have to define one label and then apply it to all your yes/no variables.
• You can define more values in a label than you have in your variable. For example, if you have some yes/no questions and some yes/no/maybe questions, you can define a yes/no/maybe label and apply it to the yes/no variables, provided "yes" and "no" are coded with the same numbers.
• Each variable can have only one variable or value label applied at a time.

Notes

Variable and value labels go a long way toward documenting your data. Sometimes, though you may need or want to include information that you wouldn't necessarily want in a label. Information on how you constructed a variable and in which do-file you did it or reminding yourself of skip patterns are examples. This is where variable "notes" are useful. Notes are not displayed in any output by default, but are a good way of keeping a record of what you did. The only drawback to using notes is the fact that only Stata has them; other packages like SAS or SPSS do not have notes, so if you convert your data, the notes will not be converted as well (variable and value labels are, though). You can have a total of 9,999 notes in your data and you can have more than one note for any variable.

Describe and Codebook

The describe and codebook commands provide information about your data. Codebook provides much more detail than describe, but you will probably use describe more often.

The describe command lists some useful information, but sometimes you need even more detail about the variables. For this, we have the codebook command.

Examining Your Data

Now that you have some data loaded, you’ll want to take a good look at it to make sure it was read in correctly and that it was what you expected.  There are four basic commands you can use to do this:  “list,” “browse,” “tab” and “sum”.