APSF partition map corrupted - What options?

[Omega Mac]

So I mistakenly allowed 10.13.5 install on a machine which looks to have corrupted the partition map.

Diskwarrior - I may consider, but right now it can't rebuild APSF 10.13, until they are able to get full documentation from Apple, accoring to their website, the next release will have that functionality.

I tried iBoysoft but that crashed while reading the drive. Not a ringing endorsement.

What other options are there or is it watch and wait for Diskwarrior?

Thanks in advance for advice and help as always!

 

[chscag]

Any backups? Without a good backup, you'll probably need to erase, and format the drive. The Alsoft site (Disk Warrior) does mention some warnings about DW 5.1 and whether or not it can deal with rebuilding APFS. You can hang on and wait for the next version of DW but I suspect that it won't be any time soon.

 

[Omega Mac]

Any backups? Without a good backup, you'll probably need to erase, and format the drive. The Alsoft site (Disk Warrior) does mention some warnings about DW 5.1 and whether or not it can deal with rebuilding APFS. You can hang on and wait for the next version of DW but I suspect that it won't be any time soon.

Yea I have a backup (CCC) but not sure how recent maybe a few weeks or month+. I always backup right before any OS updates, but this one get set to 1 hour in error!

I may not have lost much. Recovery features would probably get me what I need but there is some VM software i need to use from time to time.

So I guess really it seems APFS has no documentation (from future comments I've read) and is really a big BETA... had I known I would never have upgraded to HS - double doh!

 

[treekram]

If you can post the results of diskutil list (enter the command in the Terminal app), that may help.

 

[Omega Mac]

If you can post the results of diskutil list (enter the command in the Terminal app), that may help.

Thanks for that suggestion. I’ll try follow up tomorrow.

Further, checking out Techtool Pro 10 webpage, I spot this disclaimer at the very bottom

“In most cases, data recovery is not possible on Solid State Drives (SSDs).”

WTF! Is this a typical data recovery scenario for SSDs??!

 

[chabig]

Physical hard drives crash when the flying read/write head fails and ploughs into the disk. Data remains magnetically encoded on the disk and can be recovered sometimes. SSDs are completely different. An auto analogy would be that if an internal combustion engine throws a rod you might rebuild the engine, while an electric vehicle doesn’t have piston rods. (Perhaps not a very good analogy)

 

[Omega Mac]

Physical hard drives crash when the flying read/write head fails and ploughs into the disk. Data remains magnetically encoded on the disk and can be recovered sometimes. SSDs are completely different. An auto analogy would be that if an internal combustion engine throws a rod you might rebuild the engine, while an electric vehicle doesn’t have piston rods. (Perhaps not a very good analogy)

Yea I’m familiar with magnetic aspects but considering SSD have high shock tolerance, no moving parts and has been sold as all round more reliable perhaps I have been lured into missing data loss pitfalls. I assumed the integrity of the data would be higher.

I dont think it’s SSD failure but merely a partition map corruption. Surely the remaining data is recoverable. I guess I’ll find out soon enough - sizing up which recovery software is the best.

I sense Diskwarrior may be the best for recovery.

 

[chabig]

In my opinion it’s probably not worth your time. I’d just restore from backup and move on. Your system was probably messed up to begin with. I’d guess that the number of successful HFS to APFS conversions is over 100 million by now, and if the installer can’t validate the conversion after it’s done it’ll fail back to HFS. There is simply no way for the conversion to end up corrupted. I think you’d be better by formatting afresh.

 

[Fishrrman]

OP wrote:
"So I guess really it seems APFS has no documentation (from future comments I've read) and is really a big BETA... had I known I would never have upgraded to HS - double doh!"

Suggestion:
Reinstall the OS, but RE-FORMAT your drive to HFS+ before you do, and then do the re-install that way.

You've "learned a hard lesson" about APFS.
What is it?

 

[TiggrToo]

OP wrote:
"So I guess really it seems APFS has no documentation (from future comments I've read) and is really a big BETA... had I known I would never have upgraded to HS - double doh!"

Suggestion:
Reinstall the OS, but RE-FORMAT your drive to HFS+ before you do, and then do the re-install that way.

You've "learned a hard lesson" about APFS.
What is it?

You tell us. The way you wrote your post makes it sound like there a some major fundamental issues with APFS that HFS never had. So, rather than being coy, why don't you detail what you see the issues are over HFS because, from what I'm reading, there are some big plusses to it over the older File System.

I get that some software app may have an issue with the underlying file system when doing recovery, but, TBH, that should always be anyone's last gasp attempt - backups of any sort should always be your primary recovery point.

In addition, I was under the impression that any startup disk was automatically upgraded to APFS with any 10.13 release, and the OP didn't state (from what I could see) if they wanted to put 10.12 or 10.13 back on.

 

[chscag]

Thanks for that suggestion. I’ll try follow up tomorrow.

Further, checking out Techtool Pro 10 webpage, I spot this disclaimer at the very bottom

“In most cases, data recovery is not possible on Solid State Drives (SSDs).”

WTF! Is this a typical data recovery scenario for SSDs??!

It's the same disclaimer made by the folks at "Data Rescue" (Prosoft Engineering). The reason is because of TRIM. However, there are data recovery services who state they can recover data from a SSD. Of course that would be a last resort as the expense is high. Which is why backups are so important.

 

[treekram]

Thanks for that suggestion. I’ll try follow up tomorrow.

Further, checking out Techtool Pro 10 webpage, I spot this disclaimer at the very bottom

“In most cases, data recovery is not possible on Solid State Drives (SSDs).”

WTF! Is this a typical data recovery scenario for SSDs??!

For a SSD where the OS screwed up (vs. a problem on the SSD itself) ... On a HDD, performance is helped if the data in a file are in contiguous blocks (the head(s) don't move that much). For a SSD, that's no longer the case. A couple reasons why data may not be in contiguous blocks is wear-leveling - if you have a file that never changes it may not stay on the same physical block - instead moving to a block which has had more erases/writes to it; or it may be that to increase performance, a file may be split into different NAND cells. A lot of data recovery on HDD's rests on files being in contiguous blocks. It also may be that information on the mapping between the OS file table map and the SSD block map is not something that SSD controller manufactures make public - that's just speculation on my part.

 

[IowaLynn]

Installed or upgraded? SSD use ECC. The way file changes are written is new. Also not having to write full cell pages. You have CCC hopefully bought latest upgrade. You can boot from there.

No mention of external data drives. Or your Mac. I have MacBook and a USBC dock with 256GB SDXC card for backup. CCC you could have a recovery partition. I recommend upgrading to 5.1 beta https://bombich.com/blog

Not sure if it really is partition or journal or failed SSD.

 

[Honza1]

In my - surely limited - experience SSD do not seem very recoverable. Backup seems the only reliable solution.
I had one SSD die completely (not readable at all) and two corrupted (one HFS and one APFS). None of the SSDs could be corrected by Disk tools or other tools available to me. I had (over lot longer) had more hard drives failed and I guess that 50+% of corrupted systems (HFS, NTFS, ...) could be recovered.
On my 2017 MBP SSD failure means hand system to Apple and let them replace significant part of the whole computer. Supposedly they can recover the data from SSD through some proprietary port of the SSD itself... Hm.

 

[MIKX]

In future Hi Sierra I will be installing to a spinner HDD in HFS +then CCC that drive to another external spinner hdd as a backup ( I always have two backups of the same intallation ) and then . . . again use CCC to restore to my M.2 NVMe ( drive) .. .
Of course that will be when I am satisfied that NVMe booting is proven & reliable.

No APFS for me ( yet ). There have been way too many problems reported in the MR forums.

PS : Do you have TRIM turned ON ?

 

[chscag]

PS : Do you have TRIM turned ON ?

TRIM is automatically on for Apple branded SSDs.

 

[Omega Mac]

Thanks for all the feedback.

Got Disk Warrior, popped it on El Cap machine didn't see the drive properly.

Then tried to install it on a High-Sierra machine and I can't get past the security issue - still waiting for support. Not making much progress.

I've had a few HD's go in my time and usually always have a backup, in this instance it's a few months old, but I probably haven't lost too much (most recent work small files) been using the cloud for a change since working across multiple machines so there is that saving grace!

My current primary machine is backed up too 3 CCC HD's, one of which has a partition for Time Machine - got whacked by too-many-corpses not long ago - this however has been way more fatal due to update.

Not sure about TRIM.

Years ago on first SSD installed that program some kid made to turn TRIM on but since further SSD upgrades I didn't bother with TRIM... probably should have run some maintenance on the drive a year ago or so, before High Sierra or Sierra.

@Fishrrman - No lessons required learning here, this was my own late night human error - I mistakenly clicked "try in one hour" option, then went to sleepily to find my machbook pro had been transformed into a sack of coal.

My regime is to always do a full backup using CCC before any macOS updates.

What I didn't know was how un-recoverable SSD's are, since my regime aims to avoid being in an only a recovery scenario, I'd never hit this issue before. So it's spinning platters for all my backups (as is the case).

 

[grobik]

Hey Omega, I had a similar issue with a MacBook. APFS, split drive into to "Mac HD" "Recovery" "Preboot" & "VM".

I was easily able to boot into target disk mode. Which allowed me to see them on another machine running sierra. I did have to use terminal though to boot the "Mac HD" volume with all my data. This article covers the mounting process. Then was able to manually copy data and then reformat.

I've also heard if you have another Mac running high Sierra, and mount the affected Mac in target disk mode to it you can run a repair disk in disk utility and that should in theory fix it. But I was too far in the process to try that.

Good luck!

 

[Omega Mac]

Thanks for that pointer, which leads to the previous request to post for distil output, below appears to the be the borked SSD.

+-- Container ERROR -69808
| ======================
| APFS Container Reference: disk4
| Size (Capacity Ceiling): ERROR -69620
| Capacity In Use By Volumes: ERROR -69524
| Capacity Not Allocated: ERROR -69524
| |
| +-< Physical Store disk3s2
| | -----------------------------------------------------------
| | APFS Physical Store Disk: disk3s2
| | Size: 999995129856 B (1000.0 GB)
| |
| +-> No Volumes​

 

[treekram]

In Terminal, if you can run the following command:

diskutil info /dev/disk4 | grep Read-Only

It should come back with two lines. Look at the Read-Only Media line - it will say "Yes" or "No".

Repeat the command, substitute disk4 with disk3 and then disk3s2.

These commands should all return "Yes" or all "No". If it doesn't, that would be unusual. On a properly functioning SSD (with corrupt data or not), it should be "No". If a SSD has gone into read-only mode, it will read "Yes". If the SSD has gone into read-only mode, it's because it detected some fault and to prevent further (if any) data corruption, it went into read-only mode and there's a good possibility that whatever fault it was is what borked the ability to properly mount the disk. If it isn't in read-only mode, then it's more likely the OS or install program screwed up.

The -69808 error in the same context as yours doesn't show up much in web searches and I also saw one article where it showed up in a HFS+ disk.

You may want to make an image of the SSD - if the SSD is not in read-only mode, as you try to fix it, you may be further corrupting the data. Or if the SSD is in read-only mode, it may be that transferring the data, byte-by-byte to a new working SSD will fix the issue (although I wouldn't get too hopeful of that). The problem is that with a 1TB drive, working with an image is going to be cumbersome. Also keep in mind that writing a full image to a new SSD may take a substantial part of it's life - for example on a Samsung 1TB 850 Evo, if it does write every byte to the disk (don't know how it would handle 0-value bytes), you're using 1/150 of the SSD's life (newer Samsung and Crucial models have substantially higher endurance rating, but this example is just to give an idea of what this entails). If you want more information on imaging the disk, put a post to that effect.

 

[Omega Mac]

In Terminal, if you can run the following command:

diskutil info /dev/disk4 | grep Read-Only

It should come back with two lines. Look at the Read-Only Media line - it will say "Yes" or "No".

Repeat the command, substitute disk4 with disk3 and then disk3s2.

diskutil info /dev/disk4 | grep Read-Only

Read-Only Media: No
Read-Only Volume: Not applicable (no file system)​

diskutil info /dev/disk3 | grep Read-Only

Read-Only Media: No
Read-Only Volume: Not applicable (no file system)​

diskutil info /dev/disk3s2 | grep Read-Only

Read-Only Media: No
Read-Only Volume: Not applicable (no file system)​

 

[treekram]

diskutil info /dev/disk4 | grep Read-Only

Read-Only Media: No
Read-Only Volume: Not applicable (no file system)​

diskutil info /dev/disk3 | grep Read-Only

Read-Only Media: No
Read-Only Volume: Not applicable (no file system)​

diskutil info /dev/disk3s2 | grep Read-Only

Read-Only Media: No
Read-Only Volume: Not applicable (no file system)​

So what this says is that your SSD did not go into read-only mode which makes it more likely that the OS screwed up writing to the partition table for the APFS container.

I mentioned about whether you want to make disk image of your SSD before proceeding in my previous post.

It doesn't sound like it's worthwhile to you to go to a data recovery service but you are willing to try data recovery software. As you had found out, you will need software that can deal with APFS disks. If you have an encrypted disk, you will need software that can deal with that. If you haven't already, you can try First-Aid in the Disk Utility app. You can run that in Recovery Mode (if that still works) or from running High Sierra from an external disk (which looks like what you did to get the information you posted).

 

[Omega Mac]

So what this says is that your SSD did not go into read-only mode which makes it more likely that the OS screwed up writing to the partition table for the APFS container.

I mentioned about whether you want to make disk image of your SSD before proceeding in my previous post.

It doesn't sound like it's worthwhile to you to go to a data recovery service but you are willing to try data recovery software. As you had found out, you will need software that can deal with APFS disks. If you have an encrypted disk, you will need software that can deal with that. If you haven't already, you can try First-Aid in the Disk Utility app. You can run that in Recovery Mode (if that still works) or from running High Sierra from an external disk (which looks like what you did to get the information you posted).

The borked SSD is in a ext usb housing.

Connected to a Mac mini with High Sierra.

That Mac mini is booting from an external SSD drive. The two internals are clones of the external. I haven’t gotten time to installing the SSD internally. All a byproduct of being hit with Too Many Corpses on 10.13.4 update.

Performance is dandy.

Q: How do you suggest making a disk image of the borked HS install. Are you suggesting using something like Carbon copy clone?

 

[treekram]

The borked SSD is in a ext usb housing.

Connected to a Mac mini with High Sierra.

That Mac mini is booting from an external SSD drive. The two internals are clones of the external. I haven’t gotten time to installing the SSD internally. All a byproduct of being hit with Too Many Corpses on 10.13.4 update.

Performance is dandy.

Q: How do you suggest making a disk image of the borked HS install. Are you suggesting using something like Carbon copy clone?

I wonder if there is something with your hardware or your setup that is contributing to the problems you're having.

CCC copies files. Making a disk image makes a copy byte-for-byte of your SSD. There are instructions on how to do this in:
https://www.cyberciti.biz/faq/how-to-create-disk-image-on-mac-os-x-with-dd-command/

Some notes on using dd:
You're creating a file of what is on the SSD. It might take a while. I did a test on an internal 240GB SSD on my Mini - just 650MB worth - and if the performance held up, it would have taken under 5 minutes for the 240GB SSD. Just be prepared for it to take some time. I had to do this recently on a 64GB flash drive and it took about an hour - so be patient.
1) In the diskutil listing, to reiterate what is said in the article - you're looking for the diskX that corresponds to your problem SSD - not diskXsY.
2) In step 2, you can eject the disk from Finder - it does the same thing.
3) In step 3, instead of diskX, you can use rdiskX - this should be faster. For "of=", if you want the output to be on another external disk, it would be: of=/Volumes/mydisk/directory/filename where mydisk is your external disk name as it appears in the finder. If you just use a file name, the file will be saved to your home directory. If there's a space in your directory or file name, you need to enclose the full path in double-quotes. I would use "bs=64k" as noted in the 3rd example. There's a section that says "You can create compressed disk image as follows". I would use this because if you don't use compression, the image will take 1TB for your 1TB SSD. Using compression can significantly reduce the size of the disk image file.

There's information on how to use Disk Utility to make the image. For me, I just prefer to use the command line for making a disk image. I've also seen posts from people saying that the disk/folder imaging in Disk Utility doesn't quite work - I haven't investigated this to know that's all about. It might be that opening an image for an APFS disk (being able to mount the image) doesn't quite work - I'm not sure.

So again, the idea of doing this is that you have an exact copy of what's on your SSD for the purpose of restoring it if whatever action you take to fix the SSD makes it worse. There's information in the article on how to restore to the SSD - if you use compression to make the image, you have to use the example which has the "gunzip" command to restore it to the SSD. Because you have multiple disks, I would redo the first "diskutil" list command to make sure you have the right disk number. It may change between the time you do the image and the time you restore to the SSD.

 

[Omega Mac]

Quick note, the borked SSD was pulled out of a forever reliable 2010 MBP and put in ext. usb housing to connect to mac mini for diagnostic purposes.

Thanks for extensive post - I am reading now.