Windows Server 2008 R2 (Metafile RAM Usage)

[DaRcCrysys]

I have done quite a bit of research over the past couple of days regarding this issue and I have come to a stonewall. Here is a basic description of the situation.

I have the Three Core VDS (Unmanaged) package, with an addition 4096MB (4GB) of ram, hosted through Chicago running Windows Server 2008 R2 x64. On this server I run the following:

Feed The Beast Ultimate 1.1.2 (Minecraft 1.4.7)
McMyAdmin 2.4.9.4 Professional

For a while now I have been having issues with the VDS nearly running out of RAM. I recently contacted the NFO support team with this issue and was greeted very quickly with some recommendations to analyze the problem. I found these recommendations very helpful.

1) I was recommended to download RAMMap and perform a general analysis of my VDS's RAM usage
2) I was recommended to allow the Windows Server to manage the VM Pagefile rather than using a manual setting (changing this has made no difference int he system performance)
3) I was told I should post here for further assistance

Now as i stated earlier I have done some research (around 17 hours to be exact), and all have come back to one conclusion...

This is a known Microsoft Issue, was supposedly corrected in Server 2008 R2 and Windows 7, but is still listed as having the same issue that was "patched", and Microsoft is reported to be charging $100 for a fix to the issue.

I am hoping to fix this issue WITHOUT having to remove my entire server, swap to a Linux installation and then try to feed the server back into the OS, without losing date, corrupting files, or having an overly extended downtime.

As always, and/all help is greatly appreciated. Thanks in advance.

[soja]

What does your memory usage look like? I have had my linux VDS run out of RAM and the results are terrible(it locks up).

Is it possible to lower your allocated RAM in minecraft temporarily to help diagnose the issue? Do you watch remote desktop constantly when you think the server will be running out of RAM? A simple task manager window should tell you if you are about to run out of RAM.

[DaRcCrysys]

As stated in my initial posting, the RAM usage is the entire issue, specifically the metafile portion.

I have ran my Minecraft launch file using 1GB, 1536MB and 2GB of ram, all have the same result, metafile eating up all the VDS's RAM.

And as for watching the VDS, constantly, I have only done this once, usually i check back nor and again over a period of several hours or days checking on the usage, although it seems to run the RAM usage to nearly red line status within about an hour in most cases. I use task manage, performance monitor, and RAMMap to monitor this situation, sometimes just one, sometimes all three, depending how much time I have to devote to monitoring the system directly.

[soja]

Ah, I misunderstood the problem from your OP. Thanks for clarifying.

I am looking up things related to your cause, and possible fixes. Can you add me on steam to speed this up? Check my signature for the link. If we arrive at a solution for you I will update the thread explaining how it was fixed.

[DaRcCrysys]

So far, the only solutions I am seeing (based on all the threads, and articles that I have found) is to

a) Pay Microsoft $100 for a manual fix to implement a manual limiter on the amount of RAM that can be cached (now the charge mentioned above, is based on replies and postings to articles, I have not contacted Microsoft and received this request from payment directly)

b) Supposedly if I updated my server to Windows Server 2008 R2 a hotfix was applied for this OS addressing the issue (the issue now becomes that, I am already running Windows Server 2008 R2 and encountering the same issue said to have already been patched), reference now suggestion a

c) To move away from a non-server based Windows OS (i.e. Windows 7), or a Linux based operating system (i.e. CentOS (RHEL Branch of Linux) or Ubuntu

In a way I feel I should have gone with Linux in the first place as it offers better security, a more lightweight and cleaner code base, and in my personal opinion has a higher flexibility than Windows ever has, but I was looking for ease of use at the time to get my server back online for my members, enter my current situation...

---EDIT---

I also added you on steam for ease of communication where needed.

[Edge100x]

Can you send us a screen shot of the task manager "Performance" page, for reference?

Does restarting the process make any difference?

Is the memory attributed to any specific process?

[soja]

John, the "metafile" was consuming the ram, something only seen through RAMMAP as far as i could tell. Task manager showed all ram being consumed, but no process visible in the processes tab.

Using bits of this guide: http://blogs.technet.com/b/mspfe/archiv ... emory.aspx

we determined it was a popular minecraft plugin "dynmap". His dynmap folder was 150GB and was frequently being requested by its built in web server, and windows began to cache much of this information.

I haven't talked to the OP today, but last I heard he was altering his dynmap config to get the file size down, and going to use a third party web server for the files.

[Edge100x]

Switching to Linux is also a good plan, long-term, as is using SSD (if it's not already being used). dynmap produces tons of small files, like Minecraft itself used to, and ext4 on Linux handles lots of small files much better than NTFS on Windows.

[DaRcCrysys]

So, I have rendered the primary world in lowres mode, using the lowest settings possible for renders per tick, and chunks per render. The RAM seems to have calmed down some, but in the long run is very slowly running the metafile and RAM back up, although not at the phenomenal rate it was before.

To correct one of two minor details of a previous post, the file was only about 105GB, although still very large for a set of image files... Also we were planning to keep the internal web server going, a static map was not really our intention as our users rely on the dynamic rendering we supply through dynmap.

To another point, I had looked at using Linux, specifically CentOS because I am more familiar with RHEL based OS's or RHEL branch OS's than anything else. The issue with changing the OS platform, was that we still had a fairly large size of files (12.14GB) to move between our current server and an offsite location for storage while we lay in the new OS, and get it configured, and then try to restore the files to the new system. Now being as this is hosted through NFO I don't exactly have the ability to walk up, pull the hard drive and use copy and paste, or a direct drive clone software...

So, I am possibly still looking at converting over to CentOS for our operating system platform, and maintain a lowres map so as not to eat up the system RAM. Any suggestions how to send all this data back and forth so we don't lose anything would be appreciated. Thanks in advance!

[Edge100x]

I recommend using the SSD storage option to make copying the files easier. If you add an SSD to your package, then compress your files into a .zip and put that onto the SSD, it will be remain available across the OS reinstall (as long as you make sure the checkbox says to retain the SSD contents). The only (mildly) tricky part is making sure that your chosen version of Linux has the ability to mount NTFS partitions, but I believe they all do now.

I do still recommend backing up your files externally, though. Backups are always a good idea.