JeremyBlum.com

v3.0

Tutorial 11 for Arduino: SD Cards and Datalogging

| 61 Comments

This is my first in a series of more advanced Arduino tutorials. Now that we’ve covered the basics in tutorials 1-10 (you have watched them all right!?), it’s time to start pursuing some more complex projects! In this episode, we’ll utilize an SD card shield from cooking-hacks.com to create a datalogger. First, I’ll walk you through the process of reading from, and writing to an SD card. Then, we’ll utilize some of our knowledge from tutorials 4, 6, 7, and 8 to add several sensors that we can poll periodically using our datalogger. The logger will record sensor values to its SD card in a useful format so that we can later use a computer program like Microsoft Excel to visualize the data. Download the schematics and source code first so you can reference them while you watch the video. For the first time, I’m using fritzing to draw the circuit schematics. So, take a look at them, and let me know what you think!

 

You can download the files associated with this episode here:

GNU GPL License Distributed under the GNU General Public (Open-Source) License.
Please Attribute and Share-Alike.

61 Comments

  1. Hi there Jeremy

    Just wanted to say that I loved the tutorial you made, really helpful, I had already bought a freeduino and knew some arduino stuff but needed I2C communication, that how i found your tutorials. Just to give you some relative info, I’m studying mechatronics at CPUT in south africa and Im busy on a final year unmanned ground vehicle that can climb stairs and feedback sensor data to the user via a remote desktop. I dnt know if you’ve heard of a roboard, but its an incredible piece of engineering and you should look at it if you can.

    Thanks again, please keep the tutorials coming

  2. Great tutorial! Thank you. You are doing a very good job. I am working on my own data logger but I have some trouble with the library. You mentioned making a tutorial on it. It would be awesome! Looking forward to your next tutorial. Greetings Norway

  3. Fantastic work Jeremy. Real examples help so much.
    I have a question, if you can spare the time. I would like to record data to a .csv as shown, but only if a digital line changes state (Low>High). Would the Arduino have enough processing power to transfer data to SD at say 500 lines/second? I would be recording RTC data and one Analog input, plus controlling a two digital outputs at a user determined point.
    Thanks.

    • The atmega runs at 8MHz if I recall correctly, so it should be possible to record tat that speed. But, it might be possible to do it with the arduino programming language. Programming in C or assembly would give you much more overhead to capture data at that speed.

  4. How do I get the program file to open or copy it over to the Arduino program?

  5. I wrote the library name inside angle brackets, but somehow it disappeared. There is probably some kind of html trim going on. Anyways, I was talking about the Time.h library. I think you mentioned making a tutorial on how to track date and time too. That would be great!

  6. Hi nice blog and nice tutorials…..
    i’m SD data logging with my arduino uno.. I’m using onewire dallas 18B20 temperature sensor.. arduino compiler shows some error..can u de bug it plaease…..can i have ur email id or tell me how can i send the code i used in this….

  7. I enjoy your tutorial. Will you have a vedio on using Arduino with an AHRS such as Sparkfun 9 Degrees of Freedom – Razor IMU or a similar product?

  8. These are extremely valuable lessons. I cannot wait for your next video on interfacing a GPS. Do you han any plan to have a video on filters as you mentioned in your lesson five (servo motors)?

  9. Hi nice and very valuable work…
    Im a beginner with Arduino and your tutorial helps a lot, keep up the good work.
    And for my project i need to control two 12v dc motor,
    They should rotate in both directions and stopped, i thought of using two DPDT relays to change the polarity to change the motors rotation and another two SPST relays to stop the motors, but im little confused about connecting and controlling the relays by Arduino, could you help me in this problem?

  10. hey a wonderful way to put things down … was wondering if i cud get a schematic on the sd card module ( making in wud cost a lot less u guess )
    thnak you

  11. hi.. Thanks for the tutorials.. Its very useful to me..
    i have question for you..
    I am now currently studying about RTOS for my final project.
    Can arduino IDE use RTOS kernel, like freeRTOS?
    then how if this project implemented with RTOS??

    thank you..
    Indonesia

  12. You are great mate..

  13. Hi Jeremy,
    Great series! I really enjoy watching them.
    I have a question you might know the anwer to. I want to make a device the measure the rising of dough at different temperature. I was planning to put the dough in container with straight walls and just measure, from above, the distance to the dough. I come across two types of distant sensors, ultrasonic and IR. Which one would you think to have the highest sensitivity? They are about equally priced.

    Regards, Rutger

  14. Hi Jeremy, great blog!

    I am trying to append a float (temperature) to a dataString and writing that to an SD card on my Arduino. The compiler is giving the following error: call of overloaded ‘String(float&)’ is ambiguous.

    The offending line of code is: dataString += String(temperature);

    Was wondering if you could suggest a straight forward or easy to understand solution for a non-programmer. Thanks!

  15. Jeremy,
    Fantastic tutorials!

    Why did you use the cooking-hacks SD shield? The price in the US is twice what is charged by Sparkfun.

  16. I’m curious to know if I could implement this. However, instead of removing the SD card and putting it into my computer, I would like to transfer the data off of the SD card using one of the wireless shields. Is that even possible? At least with an appreciable amount of speed? A 24 hour log file (at a higher sampling rate, say 250 samples/s) would create a 100+MB file I believe.

  17. Sample rate @ 250 samples/s = 4 milliseconds between samples. If each sample was a single byte and the serial transmission protocol required 10 bits of transmission per byte, the theoretical transmission rate required would be 2500 bits per second. Figuring in the real capacity of a radio link, practically you would want 19200 bps rate or higher. At that speed, you would not need the SD card except to buffer when the radio link was not functional.

  18. .,hello..can you make a tutorial combining the analog inputs like using ir sensor and then sending the value to a gsm phone via sony ericsson as a modem?..

  19. Hi Jeremy,
    Your videos are fantastic. They have helped me a lot. I am myself working on a small Arduino project and occasionally have difficulties.

    I was wondering if you could shed some light on this.
    I am using the microSD card on the official Arduino ethernet shield to store some time data and temp data. I have setup a website in php/javascript to display temp-time graph from a text file. It works from any remote location such as . But it doesnt read from the ethernet sd card which is something like 179.xx.xx.x/data.txt.

    And on a different and probably complex topic. Is it possible to analyse the input data in real-time and send an email if the data point meets a certain condition?

    Once again, keep up the good work.
    Cheers

  20. hi im beginner in arduino. this is nice blog and nice tutorial. u helps me lots on many ways. here i have a little bit question on arduino with data logger. its that ur data logger program compatible with arduino ethernet? and how to program ethernet with LM35Dz temperature sensor. hope u will helping me.. tq jeremy

  21. Hi!

    Great tutorials! I watched some of them and they are just great!!

    Just a comment, in case it can help. When you read the number from the file, you convert it from string to integer. That way is ok, but, wouldn`t it be easier if you did

    refresh_rate = 0;
    while (commandFile.available())
    {
    float temp = (commandFile.read() – ’0′);
    refresh_rate = refresh_rate*10 + temp;
    }

    I think this would work as well, and you don’t have to figure out the number of digits and so on…

    Again, great tutorials! Thanks,

    Paco Lacon

  22. Hi, I was wondering if you could tell me where can I find the SD library. Thank you very much.

    Lukas

  23. Hi,Can you please give example on how to get data on a sd card file with .csv extension and add the value as a beginning value to be incremented from a pulse sensor.I have already set up my arduino and it has already writing data to the sd,but everytime arduino restart the pulse (ReadingTotal1 and ReadingTotal2) total reset to zero.
    my Sd file look like this (Data.csv)

    Time,Date,Reading1,TotalReading1,Reading2,TotalReading2
    1:00,1/2/2011,100,100,0,0
    1:30,1/2/2011,0,100,50.50,50.50
    2:00,1/2/2011,250,350,0,50.50
    2:30,1/2/2011,0,350,100.50,151
    3:30,1/2/2011,160.50,510.50,0,151

    I just want to get the last values (when arduino restart) of TotalReading1 which is 510.50 and TotalReading2 which is 151 as a beginning value to be increment again when the pulse sensor reads some signal
    thanks

  24. Saved my life with this tutorial, best i’ve seen on SD card use and was desperately needed to help finish an assignment and get my head round the basic functions. studying for a degree and working avg 45 hour week is not good :)
    cheers, keep the tutorials coming.

  25. Is it possible to use other pin ports?!

  26. Hi i buy this shield
    http://www.sparkfun.com/tutorials/172
    but when try to program, anything work maybe can you help me?
    i need make a datalogger and really i dont understand why this shield dont work on my arduino uno
    thanks nice tutorial

  27. I like these videos and this one in particular since I’m working on a similar temperature, pressure, humidity, & light level logger. :)

    If you planning and future videos I’d love to see one on a minimal arduino setup similar to this one: http://todbot.com/blog/2009/05/23/arduino-chip-sticker-label/ and/or optimizing battery life with the arduino sleep functions.

  28. if I do not use the arduino shield sd recognizes that not connected sd, but if the shield is not load because it says nothing on the screen of serial communication is blank

  29. Looking forward for your next tutorial. Very exciting indeed. My request if you could create tutorial on controling motor using PS2 controller. Very appreciate your contribution. Thanks.

  30. Hey Jeremy,

    I successfully was able to use both your write and read/write examples at one point..I then attempted to do the data logger example with a potentiometer feeding into one analog input…the attempt was unsuccessful. The problem is now when I attempt to revert back to the first example (print Hello) I am getting a card failure and could not open log file message..is it possible to fry the microSD card?

  31. Hello,

    I run Arduino 1.0 on Windows 7 SP1 and tried to run the programs that are in the download pack of this episode. The SD_Read and SD_Read_and_Write run fine (by run I mean that I hit the “Verify” button to start compiling the code). When I try to run the Datalogger.pde, it first “complains” that Wire.send and Wire.receive are now called Wire.write and Wire.read, respectively. Then I corrected that and tried Verify again.

    Now the line “Wire.write(0); //Ask for Register Zero” is marked in yellow, the line at the bottom turns orange and reads “call of overloaded ‘write(int)’ is ambiguous”.

    Below it reads:
    ” datalogger.cpp: In function ‘void loop()’:
    datalogger:98: error: call of overloaded ‘write(int)’ is ambigous
    D:\arduino-1.0\libraries\Wire/Wire.h:55: note: candidates are: virtual size_t TwoWire::write(uint8_t)
    D:\arduino-1.0\hardware\arduino\cores\arduino/Print.h:49: note: size_t Print::write(const char*)”

    My Arduino 1.0 is the standard unpack, but I also included the OneWire and TinyGPS libraries. At the Time of compiling, I had “Arduino Mega 2560 or Mega ADK” selected in Tools->Board.

    PS: Which pins are used for the SD card on a Mega 2560?

  32. i want to send the data on the Sd card via a cellular network maybe the arduino cellular sheild can this be done?

  33. Hi jeremy I`m having a little problem with my datalogger. The problem is that when I read from the serial port it only logs 2 lines, then appears some random symbols and I dont know how to solve it, can you help me with?

    #include

    #define LOG_INTERVAL 2000 //intervalos de medicion para el guardado de datos
    #define ECHO_TO_SERIAL 1 //datos al serial port, cada una medicion guardada, uueniada al serial port
    #define WAIT_TO_START 0 // hay que darle un comando aaduino para que empieze a tomar datos, solo basta oprimir una tecla

    #define tempPin1 0 //sensores de temperatura aƱadidos a los pines analogicos 0,1,2,3 y 4
    #define tempPin2 1
    #define tempPin3 2
    #define tempPin4 3

    const int CS_Pin = 10;
    File logfile;
    long id = 1;

    void error(char *str)
    {
    Serial.print(“error”);
    Serial.println(str);
    }

    void setup(void)
    {
    Serial.begin(9600);
    Serial.println();

    if(WAIT_TO_START == 1) {
    Serial.print(“Pulse cualquier tecla para Iniciar”);
    while (!Serial.available());
    }

    Serial.print(“Iniciando SD”);
    pinMode(CS_Pin, OUTPUT);

    if (!SD.begin(CS_Pin)) {
    Serial.println(“Error en la tarjeta o no esta colocada”);
    return;
    }
    Serial.println(“SD OK”);

    File logFile = SD.open(“LOG.csv”, FILE_WRITE);
    if (logFile)
    {
    logFile.println(“, , , ,”); //Just a leading blank line, incase there was previous data
    String header = “ID, Light, Temp, IR1, IR2″;
    logFile.println(header);
    logFile.close();
    Serial.println(header);
    }
    else
    {
    Serial.println(“Couldn’t open log file”);
    }

    }

    void loop ()
    {
    int tempReading1 = analogRead(tempPin1);
    int tempReading2 = analogRead(tempPin2);
    int tempReading3 = analogRead(tempPin3);
    int tempReading4 = analogRead(tempPin4);

    float voltage1 = tempReading1 * 5.0 / 1024.0;
    int temperatureC1 = (voltage1 – 0.5) * 100;

    float voltage2 = tempReading2 * 5.0 / 1024.0;
    int temperatureC2 = (voltage2 – 0.5) * 100;

    float voltage3 = tempReading3 * 5.0 / 1024.0;
    int temperatureC3 = (voltage3 – 0.5) * 100;

    float voltage4 = tempReading4 * 5.0 / 1024.0;
    int temperatureC4 = (voltage4 – 0.5) * 100;

    String dataString = String(id) + “, ” + String(temperatureC1) + “, ” + String(temperatureC2) + “, ” + String(temperatureC3) + “, ” + String(temperatureC4);

    File logFile = SD.open(“LOG.csv”, FILE_WRITE);
    if (logFile)
    {
    logFile.println(dataString);
    logFile.close();
    Serial.println(dataString);
    }
    else
    {
    Serial.println(“Couldn’t open log file”);
    }

    //Increment ID number
    id++;

    delay(2000);
    }

    • what arduino do you use for it???
      I do something like that, I use Arduino uno SMD edition and I have the same problem

  34. Hi there Jeremy

    I try to build this circuit but I have this Temperature Sensor (DFRobot LM35 Linear Temperature Sensor) That I can do to Arduiono Mega 2560.

  35. Hi!!!
    Somebody have some problem with arduino uno smd edition???
    I can do any program greater than 12000 bytes becuse the arduino finish hanging… Please some help

  36. Thanks Jeremy.
    Great website and very great job.

Leave a Reply

Required fields are marked *.

*