Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

MadXD

macrumors newbie
Original poster
Jul 14, 2014
24
8
Orlando, FL
Two days ago, my MacBook Pro (15-inch, 2017; macOS 10.15.1 Catalina) told me it was out of storage space. With a 500GB SSD, free space had dropped to 90 MB without me adding any files of significant size.

First step: Purge
I proceeded to scan through and remove unnecessary files, clearing about 50 GB. Within the next hour, the MacBook's free space continued to drop from 50GB to 1GB—without any interaction by me.

Second Step: Find the large files
Next, I searched through my SSD to see where the space was being used. The first oddity is that my Users folder shows a size of 475GB. Inside, I found only 217 GB of files [home folder @ 217 GB + Shared folder @ 8MB]. Note that my hidden files are shown and no hidden files are in my Users folder.

Third Step: Self-Diagnosis
  1. Disk Utility First Aid [in recovery mode] - Data and System partitions OK
  2. Daisy Disk [in administrator mode with full disk access] - Found 221GB of "hidden space"
Within this "Hidden Space" lies my problem. Neither Finder nor Daisy Disk can reveal the contents of this hidden space.
3) Malwarebytes [with full disk access] - Zero issues or vulnerabilities

Fourth Step: Find Help
  1. AppleCare Support - I explained and demonstrated the issue via screen sharing. Case was escalated to a senior advisor who has filed it with engineering. I provided Apple-collected diagnostic files and screenshots of system status and file sizes. Now I have to wait. I should hear back Monday about the response from macOS engineering.
  2. Reddit - No responses
  3. MacRumors Forums - Is anyone else experiencing something like this? I haven't been able to find a similar issue online.

It has nearly halted the ability to use my MacBook Pro. The lack of free disk space chokes applications that need it. My available disk space is currently fluctuating up and down, though not above 2GB (I deleted the 6.5GB Xcode app 30 minutes ago). When free space drops too low many apps and services won't run, such as: Setapp (or apps installed under it), Mail, iCloud documents, and macOS continually asks to close applications due to low memory.

Update: Four hours later, I woke up my "sleeping" MacBook Pro to find free space dropped to less than 20MB. The MacBook is warm and the fans are running.

Hidden Space is actively and aggressively taking over my Mac!

NOTE: I was on the Catalina Public Beta but have been off since the public release. The issue began 4-5 days after I updated to macOS 10.15.1.
 
  • Like
Reactions: IgorK

MadXD

macrumors newbie
Original poster
Jul 14, 2014
24
8
Orlando, FL
Can you try OmniDiskSweeper and report back?


It's the free little utility that helped me track down the culprits. No, when I say free, I mean free.
Trying now!
[automerge]1573381709[/automerge]
For the record, I do not use TimeMachine and the following tip to remove snapshots did not help—I don't have any snapshots.

 

MadXD

macrumors newbie
Original poster
Jul 14, 2014
24
8
Orlando, FL
Can you try OmniDiskSweeper and report back?


It's the free little utility that helped me track down the culprits. No, when I say free, I mean free.
Unfortunately OmniDisk Sweeper doesn't even see the hidden space. It only reports 263.8 GB of used space.
Screen Shot 2019-11-10 at 5.41.22 AM.png


Meanwhile, both macOS System Info and Daisy Disk report 498 GB of used space.
System Info reports 292 GB of "Other" space.
Screen Shot 2019-11-10 at 5.41.27 AM.png
Screen Shot 2019-11-10 at 5.45.16 AM.png

Daisy Disk reports 235 GB of "Hidden Space".
Screen Shot 2019-11-10 at 5.45.55 AM.png

Deleting any file of 'x' size causes reported "Other" and "Hidden Space" above to increase by 'x' size.
 

MadXD

macrumors newbie
Original poster
Jul 14, 2014
24
8
Orlando, FL
Nothing listed in OmniDiskSweeper is an issue. I already know the size, location and purpose of every folder and file it found.

I need to find what Omni isn't seeing... something hidden and using 234.5 GB of my 500 GB SSD.
 

MadXD

macrumors newbie
Original poster
Jul 14, 2014
24
8
Orlando, FL
For the record, I do not use TimeMachine and the following tip to remove snapshots did not help—I don't have any snapshots.


No, I do not use Time Machine
 

MadXD

macrumors newbie
Original poster
Jul 14, 2014
24
8
Orlando, FL
Actually, I posted specifically that I don't use time machine... but the post has this above it:

"This message is awaiting moderator approval, and is invisible to normal visitors."

I have already attempted to locate and remove built-up Time Machine snapshots. Unfortunately none exit, at least not that I could find using tmutil listlocalsnapshots /System/Volumes/Data
 

dsemf

macrumors 6502
Jul 26, 2014
433
107
Actually, I posted specifically that I don't use time machine... but the post has this above it:

"This message is awaiting moderator approval, and is invisible to normal visitors."

I have already attempted to locate and remove built-up Time Machine snapshots. Unfortunately none exit, at least not that I could find using tmutil listlocalsnapshots /System/Volumes/Data
I prefer to use terminal commands. They are not as user friendly but don't have assumptions built in.

The first one shows the APFS structure of your boot disk.
Code:
yyyy:~ xxx$ diskutil apfs list
APFS Container (1 found)
|
+-- Container disk1 3BEAE5D6-797A-4B0B-B86E-0DE7437ADB3B
    ====================================================
    APFS Container Reference:     disk1
    Size (Capacity Ceiling):      500068036608 B (500.1 GB)
    Capacity In Use By Volumes:   189918261248 B (189.9 GB) (38.0% used)
    Capacity Not Allocated:       310149775360 B (310.1 GB) (62.0% free)
    |
    +-< Physical Store disk0s2 58F4147A-1E3B-4808-AB9E-6662BFABC7C9
    |   -----------------------------------------------------------
    |   APFS Physical Store Disk:   disk0s2
    |   Size:                       500068036608 B (500.1 GB)
    |
    +-> Volume disk1s2 7808D72D-9C79-4996-BD9D-0A73876FF5C7
    |   ---------------------------------------------------
    |   APFS Volume Disk (Role):   disk1s2 (Preboot)
    |   Name:                      Preboot (Case-insensitive)
    |   Mount Point:               Not Mounted
    |   Capacity Consumed:         85467136 B (85.5 MB)
    |   FileVault:                 No
    |
    +-> Volume disk1s3 0256B625-8294-4038-8816-546629693DBC
    |   ---------------------------------------------------
    |   APFS Volume Disk (Role):   disk1s3 (Recovery)
    |   Name:                      Recovery (Case-insensitive)
    |   Mount Point:               /Volumes/Recovery
    |   Capacity Consumed:         528543744 B (528.5 MB)
    |   FileVault:                 No
    |
    +-> Volume disk1s4 B45C2266-58E2-403B-8E2D-9CF4280AEE45
    |   ---------------------------------------------------
    |   APFS Volume Disk (Role):   disk1s4 (VM)
    |   Name:                      VM (Case-insensitive)
    |   Mount Point:               /private/var/vm
    |   Capacity Consumed:         2148552704 B (2.1 GB)
    |   FileVault:                 No
    |
    +-> Volume disk1s5 65B255B6-EB33-4BCE-A6C8-B62A5BE88BF6
    |   ---------------------------------------------------
    |   APFS Volume Disk (Role):   disk1s5 (Data)
    |   Name:                      Catalina HD - Data (Case-insensitive)
    |   Mount Point:               /System/Volumes/Data
    |   Capacity Consumed:         176120127488 B (176.1 GB)
    |   FileVault:                 Yes (Unlocked)
    |
    +-> Volume disk1s6 A36114C6-61B1-4C82-B113-C051EB3B5109
        ---------------------------------------------------
        APFS Volume Disk (Role):   disk1s6 (System)
        Name:                      Catalina HD (Case-insensitive)
        Mount Point:               /
        Capacity Consumed:         10873937920 B (10.9 GB)
        FileVault:                 Yes (Unlocked)

This shows that I have one 500GB Container with 190GB used and 310GB free. The container contains 5 Volumes. The System volume is 11GB and the Data volume is 176GB.

If you have granted Full Disk Access to Terminal using System Preferences >> Security and Privacy, Privacy tab, you can use the following command to get a summary of the disk usage by location. Since the majority of the data usage wll be in the Data volume, make note of the Mount Point. It will normally be /System/Volumes/Data.

Code:
% sudo du -d 1 -x -c -g /System/Volumes/Data
Password:
0    /System/Volumes/Data/sw
1    /System/Volumes/Data/home
1    /System/Volumes/Data/usr
2    /System/Volumes/Data/.Spotlight-V100
2    /System/Volumes/Data/Library
0    /System/Volumes/Data/.Trashes
3    /System/Volumes/Data/System
0    /System/Volumes/Data/mnt
1    /System/Volumes/Data/.fseventsd
du: /System/Volumes/Data/private/var/db/fpsd/dvp: Operation not permitted
...
du: /System/Volumes/Data/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/0/com.apple.LaunchServices.dv: Operation not permitted
4    /System/Volumes/Data/private
1    /System/Volumes/Data/.DocumentRevisions-V100
124    /System/Volumes/Data/Users
7    /System/Volumes/Data/Applications
0    /System/Volumes/Data/opt
0    /System/Volumes/Data/Volumes
0    /System/Volumes/Data/.TemporaryItems
0    /System/Volumes/Data/cores
140    /System/Volumes/Data
140    total

There will be a number of errors for /private/var/ locations. Ignore these.

To drill down, extend the path at the end of the du command

Since Data actually has 176GB, the other 36GB is most likely in the /private/var/ area or is reserved by the operating system for pending usage.

Doing the above may do nothing more than confirm what the other tools have identified but it does provide a different approach.

DS
 

mpainesyd

macrumors 6502a
Nov 29, 2008
687
168
Sydney, Australia
There is a similar discussion about High Sierra here:
Several people found backup apps like Time Machine and Carbon Copy Cloner were the culprits, particularly if there were corrupted files. Same for Spotlight.
Also a useful discussion about AFPS using purgeable space here
 
Last edited:

mellie13

macrumors newbie
Dec 4, 2019
1
0
I have the same issue! I thought it was a perl script I was working on but that doesn't seem to be the case. This is definitely temporary storage as I also see it as categorized under other and when I reboot, it magically reclaims the space. I am running teamViewer and vmWare, and I am wondering if they are the culprits, are you running either of there tools? I have now resorted to running du hourly and diffing the files. The changes are too many so I am narrowing down the folders that change the most but that is a very difficult approach
 
Last edited:

Will Martins

macrumors newbie
Jul 6, 2020
2
0
Hi @madizin Have you received any return from apple? I'm at a very similar situation. To me appears that there are some problem with apfs and the new mount points. I notices that the storage as hidden, is the same siza of /Users , and also noticed that inside /System/Volumes/Data has a /Users folders also. When I check with "du" the / mount point , it's give me almost the size of the disk. In my case, the half of the disk is full with hidden files. Tried everythig. The last option is resinstall everything cleaning all disk ( that I would like to aviod ) , or simply reinstall macos ( impossible bc have just 4B free ) Tell us how did you solved please! Cheers
 

bousozoku

Moderator emeritus
Jun 25, 2002
15,602
1,760
Lard
Have you tried OnyX to help you clean up what might otherwise be inaccessible?

I've always found it useful, especially in the early days of Mac OS X, when it didn't always clean up after itself.

You should do some research to make sure you're okay with the outcome, but it was often able to quickly fix some problems that would have taken a lot of manual work.
 

Attachments

  • Screen Shot 2020-07-06 at 6.42.14 PM.png
    Screen Shot 2020-07-06 at 6.42.14 PM.png
    190.4 KB · Views: 571
  • Screen Shot 2020-07-06 at 6.41.33 PM.png
    Screen Shot 2020-07-06 at 6.41.33 PM.png
    169.7 KB · Views: 676

Will Martins

macrumors newbie
Jul 6, 2020
2
0
Have you tried OnyX to help you clean up what might otherwise be inaccessible?

I've always found it useful, especially in the early days of Mac OS X, when it didn't always clean up after itself.

You should do some research to make sure you're okay with the outcome, but it was often able to quickly fix some problems that would have taken a lot of manual work.

I've tried onyx already, daisy, diskinventory, cleanmac, EVERYTHING. There is no snapshots of tm . Everytime I delete something, the "others" space ( hidden space ) increases. I did a lot of research . Right now, I'm doing 2 things: backingup important files manually, and trying to make a timecapsule remote backup . Than I will try to repair the disk1 in safe mode, and if not work, reformat everything. Hope that timecapsule works to recorver from the safe mode after full erease. :(
 

bousozoku

Moderator emeritus
Jun 25, 2002
15,602
1,760
Lard
I've tried onyx already, daisy, diskinventory, cleanmac, EVERYTHING. There is no snapshots of tm . Everytime I delete something, the "others" space ( hidden space ) increases. I did a lot of research . Right now, I'm doing 2 things: backingup important files manually, and trying to make a timecapsule remote backup . Than I will try to repair the disk1 in safe mode, and if not work, reformat everything. Hope that timecapsule works to recorver from the safe mode after full erease. :(

It's sad to be at that extreme. I've not been there since the early days of Mac OS X, when the installer needed to be run 2-3 times to install everything. On the other hand, Windows 10 regularly does ugly things when Microsoft updates it. I haven't had full use of that machine since December, maybe.
 

dsemf

macrumors 6502
Jul 26, 2014
433
107
Hi @madizin Have you received any return from apple? I'm at a very similar situation. To me appears that there are some problem with apfs and the new mount points. I notices that the storage as hidden, is the same siza of /Users , and also noticed that inside /System/Volumes/Data has a /Users folders also. When I check with "du" the / mount point , it's give me almost the size of the disk. In my case, the half of the disk is full with hidden files. Tried everythig. The last option is resinstall everything cleaning all disk ( that I would like to aviod ) , or simply reinstall macos ( impossible bc have just 4B free ) Tell us how did you solved please! Cheers
/Users and /System/Volumes/Data/Users are the same location. /Users is a special link to the real location.

When I have questions about space usage, I use terminal. The output has been edited to reduce the volume.

Code:
cd /System/Volumes/Data
sudo du -d 1 -x -c -g .
1    ./.DocumentRevisions-V100
178    ./Users
18    ./Applications
0    ./opt
0    ./Volumes
0    ./.TemporaryItems
0    ./cores
209    .
209    total

df -h
Filesystem      Size   Used  Avail Capacity   iused      ifree %iused  Mounted on
/dev/disk1s6    466Gi   10Gi  220Gi     5%    487665 4882989255    0%   /
devfs           190Ki  190Ki    0Bi   100%       656          0  100%   /dev
/dev/disk1s5    466Gi  230Gi  220Gi    52%   1757443 4881719477    0%   /System/Volumes/Data
/dev/disk1s4    466Gi  5.0Gi  220Gi     3%         6 4883476914    0%   /private/var/vm

I can see that I have 10GB of system data (the OS) and 209GB of data which uses 230GB on the data volume. The du command does not authorization to look at everything in /private/var so that might be part of the missing data.

If I need more detail, I can extent the . to ./Users, etc.

DS
 

bousozoku

Moderator emeritus
Jun 25, 2002
15,602
1,760
Lard
Those of you having a problem, was your drive converted to APFS automatically, or was it formatted as APFS and your data restored manually?

I noticed problems with the conversion process on my hard drive, initially, but I formatted an SSD as APFS and had zero drive-related problems after that.
 

rmfrank

macrumors newbie
Sep 18, 2020
1
0
It's been a while since the last activity, but I've just faced the same problem. Using the terminal and the sudo du -d 1 -x -c -g /System/Volumes/Data/... command (drilling down the path with the largest size), I ended up in my library (of my account) and in the mail configuration. There is a log directory there, which was filling up.

So, in my case, the culprit is Apple Mail - but I don't yet know why that is filling the log file like mad.

The path, in my case is: /System/Volumes/Data/Users/<user>/Library/Containers/com.apple.mail/Data/Library/Logs/Mail

Within that directory, it was a log file (.txt) from the mail connected to a Microsoft exchange account ... (214Gb)
But this was not the only big one, all imap accounts are also producing large log files.
 

JackieTreehorn

macrumors 6502
May 22, 2005
491
427
Amsterdam
Let me just add to this, but I have the same problems on three different machines (2019 MBA, 2015 iMac, and 2019 16" MBP).

It's horrible, hidden space take up at least 70% of the SSD storage on all three machines without any clue on what to do to get rid off it. Apart from maybe a clean install.
 

Heath Close

macrumors newbie
Nov 12, 2020
4
0
Fort Worth, TX
I'm in the same boat... will comb through this again because I have suspected in the past that the time machine might have something to do with it.

I use grand perspective to find large files and I literally just cleaned off like 50 gigs 2 days ago. They are back.

I'll go back up and try some of the solutions for the time machine fix first.
 

Heath Close

macrumors newbie
Nov 12, 2020
4
0
Fort Worth, TX
I thought I had a virus. I even tried to reinstall but it kept failing. I tried changing virus programs. Today, I cleaned it up in 2 steps thanks to DaisyDisk. Thank you to those who pointed me to it!

Is there a way to shut snapshots off? That was the culprit.
 

Attachments

  • Screen Shot 2020-11-19 at 2.10.44 PM.png
    Screen Shot 2020-11-19 at 2.10.44 PM.png
    372.3 KB · Views: 394
  • Screen Shot 2020-11-19 at 2.13.22 PM.png
    Screen Shot 2020-11-19 at 2.13.22 PM.png
    366.7 KB · Views: 412

MadXD

macrumors newbie
Original poster
Jul 14, 2014
24
8
Orlando, FL
@Will Martins @mellie13 @Rockabilico @dsemf @Heath Close

I apologize to anyone who started helping me; or anyone looking for more insight for similar issues.

After starting this thread, MacRumors blocked my account because I had created two (an unintentional side-effect of wanting a new username). It more than a week to have my account re-activated. In the meantime, I couldn't respond or update with the solution.

Due to overly-protective and difficult-to-contact MacRumors admins, it was other MacRumors users and site guests who suffered. I ended up moving the conversation to Reddit, where I received zero responses, input or signs that my experience was able to help anyone else.

Conclusion below:
 
Last edited:
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.