Friday, August 3, 2007

Is there a Mediamax cover up?

On the last entry, I flagged up when Mediamax disabled comments on its blog and deleted those that had been left and suggested Mediamax is not wanting its new customers to know about our concerns. Mediamax's John Hood responded on this point (though not the others raised since he last commented) to say:

"Huh?! Of late our whole blog is about problems with the system. We're hardly hiding anything. We've addressed every issue brought to us by users through support and posted it on the blog for all the world to see."

Okay, well what does it currently say on the Mediamax blog? "We installed new upload servers! We’re sure you’ve seen the improvement in upload processing because we’re seeing record upload levels...."

No suggestion of problems there.

A few more representative quotes from the blog:

24th July: "Uploading continues to work much better...."

21st July: "Thanks for your patience through our major migration to a new datacenter. The worst of it is behind us.... When you attempt to download these files, an “ooops and error occurred” message will appear. We are working to resolve this problem as soon as possible..... [no update that it is still not resolved nor indication of when it will be]"

11 July: "We’ve corrected most of the issues that resulted from the transition....."

2 July: "The transfer of our IT/DB operations to an outside vendor in Chicago is complete. Of course, when a move this large occurs there are going to be a few random issues to iron out. The known issues are listed below...." [Yup, this one was telling people of 'a few random issues', then nothing the till the 11 July post suggesting the problems are corrected].

This unofficial blog shows people are experiencing serious problems with both the system and customer support.

What is most disappointing is that despite John's 'Huh?!' comment, we are learning far more through Mediamax comments posted here than from the official blog.

For example, we have learned here from Mediamax:

28 July: "We don't allow comments on the blog because we don't the manpower to manage it and respond. We're a small start up of less than a dozen people....."

31 July: "There's an issue with some bit for bit copies, that compunds this issue, a new upload of an existing file won't work unless you change the content of the file itself. Just changing the file name won't work, you need to edit something like the metadata to change the checksum of the file. A new upload should then result in a working file. We are very sorry about this and are working to resolve it but please try the above workaround in the meantime."

And here's the most amazing one. An email from Mediamax posted as a comment on the last entry. This can also be seen at:
http://www.techcrunch.com/2007/07/26/streamload-upgrade-goes-very-wrong-some-users-revolt/

---Quote begins
Hi Michael,

We appreciate your willingness to hear Streamload’s feedback. Yes, Streamload has had a tough several weeks due to two serious issues on our part.

On June 15, Streamload had a major storage problem that has caused many of our customer files to become inaccessible. They are not gone forever, but it is taking a very long time to recover all the data because of the extraordinary amount of data stored. With our current recovery infrastructure, we can “only” recover about 10 terabytes per day, but we are adding more hardware to speed this process.

Combined with that, we had another project underway to move our entire infrastructure to a new datacenter with the goal of making it more efficient and cost effective for customers. The move did not go as planned since it took longer than we anticipated and there was much more customer impact than originally estimated.

With the new data center, MediaMax is backed with an all new technical infrastructure, which is fully redundant, has more capacity, and is faster. This new transition, as challenging as it has been, will ultimately allow us to provide better service and support more customers in the future.

On that note, your reader comments are right - a better job should have been done to communicate with affected customers. Though we may not have communicated quickly or with much detail, we are attempting to actively address concerns, and are exploring ways to address this issue and improve our customer service.

Having said that, three weeks ago I took over as CEO again and will be able to monitor the service and customer needs more closely and hopefully bring back the customer satisfaction that was built since I started the company 9 years ago.

-Steve Iverson
---quote ends


It seems to me that if Mediamax knows "we can “only” recover about 10 terabytes per day" it would be a calculation of a few seconds to calculate how long we have to wait till it is all recovered, working at the current rate.

Steve knows what is going on, but the answer I received from support on 30 July, posted here previously, said: "The fix is ongoing and in progress, but I don't have an ETA on it's completion. I'm sorry I don't have a better answer for you than that."

As Steve says: "On that note, your reader comments are right - a better job should have been done to communicate with affected customers."

Let me just repeat the comment from John Hood, Mediamax Director of Customer Support:

"Huh?! Of late our whole blog is about problems with the system. We're hardly hiding anything. We've addressed every issue brought to us by users through support and posted it on the blog for all the world to see."

7 comments:

WillTRiker said...

Well, it's basically what I said in the entry before this one. The MM blog does not really reflect the amount of troubles we're still having with the system.

The only thing what posted on there are the blurbs from support staff informing us that things have been changed for the better or fixed .... allthough the "for the better or fixed"-part so far proves to be untrue. In that respect Blogs like this offer a more honest picture of the status MM is in atm.

WillTRiker said...

Oops, hit the publish button a little to early.

Wanted to remark that it's no so much of a cover up as MM only allowed commenting about 3 months after the switch from V4 to MM, but it gives one a little odd feeling. People that are new to the service might conclude from the Blog that problems have been handled and that MM is tinkering up their system while in reality the core system is still in dire need of fixing up.

Anonymous said...

I just want to know one thing. Will the files I can't download ever be available again? If mediamax would just answer this truthfully, yes or no, I'll be somewhat placated. If the answer is 'no' I won't be happy but at least I can move on and put it behind me. But 'not knowing' is really pissing me off. I suspect however that they will simply avoid answering this question forever and just keep stringing us along. If they do that they are beyond contempt.

John Hood said...

We've been clear that some files are inaccessible and that there are still some issues with uploads. Uploads HAVE improved since this the downtime from this transition. We are performing more than 100,000 a day.
We HAVE corrected most of the issues. If you look at the issues we listed in our July 2nd post at blog.mediamax.com we HAVE corrected most of those issues. And there has been significant progress in all of them.

WillTRiker said...

Give me a break John, if I, and other people from my community that work daily with MM, say things haven't really improved then you should take that as fact and no matter how much progress you think you have made on fixing things.

Some remarks on our upload board of the last 48 hours:

"Has anyone emailed MM about uploading? I just sent one in saying... "It seems the only way to upload is through the web upload. Unfortunately, even though you have installed new hardware to handle upload processing, it isn't working. I think I have about a 20% success rate, if that, in uploading and actually having them show up at all in the file manager. Most go into netherspace never to show up no matter how many times I try the same file. Only once did the upload actually disconnect itself because it encountered an errror connecting with the server."

"Has uploading taken a turn for the worse? One has shown up today."

"all those uploaded on my home adsl have shown up, only about 1/2 of those uploaded on my work's T1 appeared, so about normal for MM at least for me"

I've been saying for about 2 weeks now that MM XL hovers between 5 and 10% of the normal speed for me. For my fellow community members it isn't much different ... upload speeds have decreased 70% or more and despite what is stated on the blog your processing system STILL manages to lose masses of uploads.

Perhaps you're right and significant progress has been made ... it sure didn't come to our benefit then so as far as we are concerned MM has yet have very, very much to improve.

Do me a favour ... next time you think you have improved the upload and processing system drop me a line at Gem_hadar@hotmail.com and we will test it for actual improvement for you and let you know our findings. If it indeed has improved (prerably by a lightyear) then by all means, feel free to raise the trumpets on your blog, but quit stating things have improved while we have not noted such an improvement.

Albatross said...

Mr. Hood,

Some files!?!

How about nothing for my account for the past three weeks and just tried once again not thirty minutes ago and received again nothing more than the trite error message.

I have not been able to download much of anything since just before my last billing cycle which occurs every 21st of the month, except for last month when you billed it late on July 20th. checked yoru blog then, the old one, the dates for those July postings kept changing. Why is that Mr. Hood, perhaps so people would not know how long your problems were going on over the last eleven days of July?

A

You have corrected issues for whom? MM, or we paying customers?

Albatross said...

Mr. Hood,

One last question in reference to your statement pasted below:

""Huh?! Of late our whole blog is about problems with the system. We're hardly hiding anything."

Tell us Mr, Hood, what exactly ARE you hiding?

A