SD Card failing after some time. Possibly hardware problem?

I have yet another problem.

In my project, I have the SodaqSara 412 AFF going to sleep, and then waking up.

Not that this process is always reliable, but even if it does wake up, (and, while testing with USB output enabled, and prints the wakeup message) - it hangs at SD card initialization.

The code is :

 Sd2Card card;
  SdVolume volume;
  SdFile root;
  SerialUSB.print("\nInitializing SD card...");
  // we'll use the initialization code from the utility libraries
  // since we're just testing if the card is working!
  if (!card.init(SPI_HALF_SPEED, SD_CS)) {                                      SerialUSB.println("initialization failed. Things to check:");
    SerialUSB.println("* is a card inserted?");
    SerialUSB.println("* is your wiring correct?");                 
    SerialUSB.println("* did you change the chipSelect pin to match your shield or module?");
    while (1);
  } else {                                                      SerialUSB.println("Wiring is correct and a card is present.");
  // print the type of card
  SerialUSB.print("Card type:         ");
  switch (card.type()) {
    case SD_CARD_TYPE_SD1:                                              SerialUSB.println("SD1");
    case SD_CARD_TYPE_SD2:                                              SerialUSB.println("SD2");
    case SD_CARD_TYPE_SDHC:                                             SerialUSB.println("SDHC");
    default:                                                    SerialUSB.println("Unknown");
  // Now we will try to open the 'volume'/'partition' - it should be FAT16 or FAT32
  while (!volume.init(card)) {                                            SerialUSB.println("Could not find FAT16/FAT32 partition.\nMake sure you've formatted the card");
  SerialUSB.print("Clusters:          ");
  SerialUSB.print("Blocks x Cluster:  ");
  SerialUSB.print("Total Blocks:      ");
  SerialUSB.println(volume.blocksPerCluster() * volume.clusterCount());
  // print the type and size of the first FAT-type volume
  uint32_t volumesize;                                                SerialUSB.print("Volume type is:    FAT");
  SerialUSB.println(volume.fatType(), DEC);
  volumesize = volume.blocksPerCluster();                                       // clusters are collections of blocks
  volumesize *= volume.clusterCount();                                        // we'll have a lot of clusters
  volumesize /= 2;                                                  // SD card blocks are always 512 bytes (2 blocks are 1KB)
  SerialUSB.print("Volume size (Kb):  ");

This works for a few 100 cycles, then it fails at card.init. I can reset the board by pressing the reset button, i can flash the firmwire in, nothing will do any good.

The only solution is to physically unplug the card and replug it.

How can I debug this situation ? Thank you. Is this a hardware problem of the SD card, or is there still something the SODAQ board can do via some setting or software?