I built my midmarket practice at Microsoft Across America tours. You know, the Microsoft Big Day / Microsoft Connections morning show that runs along with TechNet, MSDN and before TS2. Not ashamed to say it, for years I would get in my car and drive around Florida handing out USB drives, CDs that I cooked myself, flyers, newsletters, business cards - I hit folks up with every cheap marketing ploy I could use to get more contacts, more touches, more sits.. I often say how I copy Microsoft in my SMB approach because I spent more volunteer hours than anyone I know watching Microsoft pitch and seeing what customers write down, what makes them tick, what makes them roll their eyes. Want to know why I am so successful? Because I saw a few thousand SBS pitches land in sand before I ever had to deliver one myself - no, I haven’t lost and no you won’t find me writing a book about it. It’s not a big secret why we’re the biggest SBS shop in the world. This is why I’ll part with gold teeth to go to WWPC even though I get $0 leads from them, life is a lesson.

Microsoft Connection ends, being the nice guy that I am I’d always help the presenter collect the surveys, give away the swag, help the ladies with their printer to the car.. the usual “in the south” stuff that you’d never imagine me doing having read this blog. So eventually I would start staying for the TS2 stuff because Indy or Sean or JJ became friends as I was spending more and more time at these events. This is where I learned about the true Microsoft Partner breakdown, one of which Mike clued me in last week, so let me share it with you.

As usual, I am talking about YOU:

Are you an SPF?

SPF - Single Point Of Failure Consultant. Usually one guy but at times can have multiple minions around him. Easy to spot because in addition to selling IT services it also abuses child labor laws and also has six other businesses to go along with it. No business card though. Is a Microsoft Partner supporting hundreds of sites worldwide but has not heard of the Microsoft Action Pack.

MO: Sell & Bail. Sell SBS, hopefully illegal, destroy the network by taking it back to P2P and move on to the next victim.

SPF’s in my mind are pretty much criminal operations to being with. You’ve encountered them, you just don’t know them because they fly at night and do not have an identity associated with them. That guy, our accountants friend, the college IT kid, are the usual names associated with them. Not IT people, not in it to build a business, in it pretty to take a quick buck and move on. I look down on these people.

Are you Riff Raff?

Riff Raff - Legitimate IT operation but not interested in excellence or keeping up with times, trends or certifications. IT shop without appreciation for business, without appreciation for time value of money, scale of business and solutions in general.

MO: Know one product, know it well and will support it till blue in the face. Will sit at an SBS box for six days while business is down without calling PSS to fix it in 5 minutes “because you don’t learn anything by letting people fix it for you.”

Most of TS2 audience. I don’t look down on these people but their business model and my business model are completely opposite and I just can’t relate to them. I don’t think they are evil, they just need to be open to more ideas and suggestions and not be so stuck in their ways.

If you sold SBS to a rapidly growing company with 50 users.. you might be riff raff.

If you still aren’t a Small Business Specialist because you don’t need a certification to tell you that you’re a small business specialist (oh, and ignoring all the benefits of it).. you might be riff raff.

If you refuse to ask for help because you are the brightest thing out there.. you might be riff raff.

If you don’t know what a blog is and are too busy to learn.. you might be riff raff.

If you spend a majority of your day uninstalling spyware instead of learning how to manage it, users and businesses that get overwhelmed with it.. you might be riff raff.

*New* From Mike: IT Lifestyle Partners

IT Lifestyle Partner – Individual who makes a decent living off IT but is not interested in growth beyond what is neccessary to sustain their lifestyle. Not overly opportunistic, not sales or comission motivated, true asset to a small business because they take it first and extend their years of experience as almost an employee and only do whats in the best interest of the customer.

MO: Have a customer for life.

I am obligated to kiss these guys asses because they make up roughly half of my partner program. The age does not seem to be a factor in this equation (it’s not just retired folks) but hakuna matata plays a big part in their professional approach. They objectively look at customers demands, recommend what makes sense for the practice and just walk away with the check for their time. You will never hear/see these guys because they do not climb to the top of the partner programs because they are not concerned about forcing as many sales as possible in order to retain their Titanium Partner status within the organization. They have no sales goals, no Ferrari dreams, no pressure; they just make their living doing whats right and are compensated for their time. This is also why most don’t know about this tier at all, because advertising and pressuring folks like this into sales goals and product tryouts just does not work.

SPF-RiffRaff-ITLifestyle Mashup

I had a baaaaaaaaad day on Thursday. It is usually when I’d ring up a few folks and head to Margaritaville to drink my troubles away, but that just isn’t possible at 1 PM. So I got in the car and went to see my buddy JJ knock out 150 PPT slides at a TS2 event – gotta support the locals community and all. The absolute best part of TS2 are the impromptu polls, I ALWAYS turn around to see what people say. So, let’s do the numbers. Well over 50 people in the theatre (usually well over 100,200 when they hold it at Waterford)

JJ: “How many of you are SBSC’s?”

5 people raise their hand. Ok, so we’ve got 45 potential riffraff in this audience. At least they came out to see the show, right, at least some investment is being made into the business. Let’s give them a benefit of the doubt.

JJ continues the deck, goes through business opportunities with System Center Essentials, explains EHS, answers Q&A about MAPS, Partner Program, SBSC exclusive benefits.. bet you there are more than 5 next time around! Anyhow, the show ends. I walk up to chat with JJ, answer any audience questions as the Official CH (Community wHore) and the SPF comes to self-identify itself.

I have at least six witnesses to this so I am not using this lightly. These people came to the event. These people sat through 4 hours of technosales pitch on stuff that is about as close as you can get to TechNet. They asked questions took notes. These are people that fix someones computers, these are people that sell servers, workstations, IT solutions. What did the SPF ask?

SPFer: Every now and then my computer goes blue and starts writing memory to disk. I see it with customers too. What would cause that?

Now most people roll their eyes and try to take a seat, quickly, while I’m pulling out my SPF rubber stamp to nail a warning to this persons head. “Inspected by Vlad: THIS INDIVIDUAL IS A CRITICAL THREAT TO YOUR NETWORK.”

So, where do you fit?

Microsoft released Exchange 2007 Update Rollup 4 last week and due to the office craziness I didn’t get a chance to talk about it. First of all, Exchange updates starting with Exchange 2007 are cummulative, meaning they include all public hotfixes/patches since RTM. Bigger download, but hopefully a more up-to-date server in many ways.

We rolled this out midweek and have not found any issues with it yet. We have actually been pretty happy with it because we have a number of users in both New Zealand and Western Australia, who tend to be ignored by Microsoft when it comes to DST updates. Well, not anymore. The patch for HMC 4.0 is in there as well.

Gotta hand it to the Exchange team, they keep on kicking out solid stuff, even if it expects 8 TB of ram for a 100 user site

[xxxxx   Vlad Mazek 8/23/2007 - the day Vlad finally caved in to the vast sucking pressure of MVP mafia]

Dear Riff Raff,

Just say no.

Say no to the paid off MVP mob force that is trying to squash any bit of truth when one of their own screws up.

Say no to the events that make you pay to watch infomercials, eat in a parking lot and then spam you.

I am not paid off to tell you to give my friends money after they screwed you, I do not have an alterior motive, I do not have a competing event/conference. I just believe in truth and value and if you think you’re riff raff there are better places to spend your money:

Spend it on true SMB community events: SBS Migration ITPRO Conference (New Orleans), SMBTN Conference (Dallas),  SMB Focus (Australia)

Spend it with the MSPU, Mobilize SMB, Great Little Book, SBS Unleashed

Spend it to send a thank you postcard from your vacation to the SMB blogging elite that calls it like it is and never asks for a damn thing in return: Susan Bradley, Susanne Dansey, Tim Barrett, Eriq Neale, Andy Goodman Blog, Andy Goodman Chat, Dana Epp, Vijay Riyaait, Chad Gross, Dean Calvert, Andy Parkes, David Mackie, David Schrag,  Steve Wright, Jeff Altman, Bill Leeman, Anne Stanton, Richard Tubb, Bill Waters, Jason Lieb, Karl Palachuk, Mark Crall, Larry Lentz, Nick Whittome, Kevin Weilbacher, Amy Babinchak and the TS2 community guys who although Microsoft employees probably spend more than anyone outside of 9–5 to locally support the SMB community.

Bring some food, drag a vendor, contribute a presentation or share notes from your SMB user group meeting.. or start one.

These are the people and events that build our community, if you like them support them. If you want to call yourself a leader try being honest about the past mistakes instead of glossing over them, try supporting and promoting new blood instead of clapping on your old social circle, try focusing on the big picture of promoting community involvement instead of nitpicking little nuances to support taking people to the wallet cleaners and locking them into your limited frame of thinking. You’re either leading, or standing in its way. I hope you choose correctly.

Bad leader,
MVP,
Community crusher
But not a sold out whore,
-Vlad Mazek

Tonight I wrap up the worst week in my professional life, bar none. I will gladly accept and live with my shortcomings in the educational, technical, business and other areas of my life, college had prepared me for endless trials and failures. I cannot accept the shere incompetence and the cascading layers of crap I’ve had to fight with this week. To sum it up in a picture:

I didn’t have a visor, I didn’t have an ExchangeDefender blade but man, this was an ugly week.

It all started with the Google buyout of Postini. Our sales just exploded, our network had to double, quickly, and it just so coincided with the v3.1 upgrade that was more than your “cosmetic” touchup of the web interface. All is well now, which is why I can write about it, so here is the story:

Sunday, August 19th, 2007

Sunday was the day when we decided to implement NDR failures for hosts that do not have a valid PTR record (reverse lookup). Simple thing, we have been testing it for a while and it showed great results. So we implemented it and for a few hours all was good. * True mark of incompetence is when you truly underestimate just how dumb you are. You’ll see how/why in a minute.

Monday, August 20th, 2007

There are many ways to issue an NDR, even more places and codes to handle it with. What did we pick? 4.7.1, tempfail that stops the message at the MTA level (the SMTP server basically). This isn’t really a reject, it’s a temporary non-delivery status that is issued during the SMTP conversation – all mail servers attempt to deliver the message again.

Roughtly by mid-morning we started fielding a ton of calls about the mail just not showing up. It’s not in our logs, its not in our system, it’s just vanishing. WTH? Eval the new rules, they check out. Eval the routing, all checks out. Run every component of ExchangeDefender in debug mode, all works.

Somewhere during the afternoon we finally look at the counters for DNS failures and realize.. man.. the reason things aren’t being seen is because we’re tempfailing them all. Wait, it gets MUCH worse.

Customers are livid at this point.

So we decide to do something brilliant, something so… genious.. that we just deserve a darwin award for it – we move the rules from the MTA to our bayesian filtering. Sound reasonable.

You generally want to handle these events at the MTA level because once the message is allowed into the system the connection can be dropped and there is no temporary failure after that. You’re just left in the water.

For the record, we had three failure codes, all 4.7.1 tempfails that would force the remote server to retry.

We used a temporary failure for the failure to look up the PTR record. We used a temporary failure for the forged PTR (PTR does not match the A or EHLO does not match the PTR). Finally, we had the failure handler that defered messages if the IP lookup failed completely (no RDNS).

Monday Night, August 20th, 2007

Moved the RDNS checks from the MTA where they should be to the bayesian scanner where they shouldn’t be. the lack of logic used at the time: At least  if we accepted it but assigned a high enough of a SPAM score nobody can complain about them not receiving messages, we will just hold a hard stance on the fact that anyone without valid RDNS is a spammer. How could this go wrong, right?

This was the beginning of the end. We had rewritten the highly efficient MTA code into the ridiculously inefficient bayesian filter that basically did a pattern match of:

Unknown: [1.2.3.4]

Tuesday, August 21st, 2007

Daily mail quarantine reports start to lag and while looking into it the disk and database performance are actually running where they are expected. So not a system or disk bottleneck. What the heck is slowing it down so much?

Apparently far too many people use SMTP to route intranet mail to their servers from their desktops. The world does not revolve around Exchange. So, each mail server stamps “Uknown: [1.2.3.4]” in the message headers, internal message headers – not the external ones. So even though the external interface had a valid PTR assigned to it, nobody keeps PTRs for internal clients. Result: 18x more false SPAM than usual.

But wait, it gets worse, much worse.

Tuesday afternoon…

Having just been cried to by yet another account I tell my guys to just yank the whole thing. I don’t want to hear about it, I don’t want to deal with it, just.. enough. Start provisioning the new servers, we’ll quadruple the size of the network if we have to but we are not bleeding away anymore money.

Provisioning work starts, new servers start spinning up all over Texas, California, Illinois and London.. Life is good. Almost too good.

And it was. The new version of CentOS (base OS we use for ExchangeDefender) comes with the new version of Perl (5.8.8). The new version of Perl (5.8.8) is not compatible with the optimized DBI packages we have for database operations.

Just to sum up: At this point customers are cancelling, partners are ready to crucify me, our new servers aren’t spinning up. But there is so much more waiting that I am just not aware of… One of my old high school buddies used to say: Shit happens. Shit always happens. Shit is happening to me right now, I just am not aware of the specifics yet.

Wednesday, August 22nd, 2007

Causality is a wonderful thing. Until it goes against you.

RDNS caused us to tempfail a lot of legitimate mail which caused customers not to receive mail. We then decided to accept the mail and instead just quarantine it. However, by doing so we just transfered the problem from an invisible but frustrating place, to a very visible but catastrophic place: now that our internal systems could account for the specifics of the PTR mess, they destroyed the auto-whitelist and IP reputation databases we handle.

Result: Packed SPAM reports again. Kill me now. As my friend Rich says: Fuck me running. I am not sure how that applies but I am sure it holds the degree of difficulty my day was about to get.

Wednesday afternoon, August 22nd, 2007

I’m out and about with Dave Sobel. We’re talking shop, every now and then I take a moment to look at my PDA to look at my credibility and SLA slide into the ocean. Great. At about 3:40 I fire off this message to team@:

This ends tonight.

At about 9 PM after the dust settles and we have a game plan I fire this one off to our partners:

Dear ExchangeDefender Partners,

I am sad to have to write this message to you but it has come to my attention that the ExchangeDefender v3.1 has received perhaps the worst satisfaction of any product release in the history of my company. Considering that I have written some of these solutions of scratch, the fact that at this stage in the product cycle we are having issues, I am more than disappointed in all that has happened.

I have a stack of 84 printed tickets and emails of complaints and issues with the ExchangeDefender v3.1, issues ranging from two week problems to the ones that we experienced with NDRs on Monday and false positive ratios on Tuesday. I have assembled my senior team and every member of the ExchangeDefender team, I have taken my staff away from training and we will work until the problems are resolved. You should expect 99% of functionality to be restored by 9 AM EST tomorrow. There are other changes being made to make sure this never happens again but you will receive a separate communication regarding that.

We are on top of it, we are working on it and we will have it fixed. Expect another update at 9AM EST tomorrow morning.

I realize many of you put your reputation on the line with us and I regret that we have done anything to shake your customers confidence in our product and in your ability to deliver it. We have the entire company working to resolve the litany of annoying issues and we will have them resolved by the morning.

Thank you.

It’s Thursday now. The backend fixes and commits are flying all over the place. I am trying to keep on top of one thing after another, trying to spot all the problems and see what gets to be fixed. I take a little 10 minute powernap and at 9 AM start writing this:

My staff and I have worked overnight and here are series of adjustments we have made to the backend as well as the issues we noticed in many of the reported messages that were sent to us via the support portal. We were able to isolate a number of issues with the client base that identified problems, we also uncovered a lot of problems with our infrastructure and this update is to let you know what we’ve done so far and what you can do.

Summary:
We have resolved almost all of the backend infrastructure problems and are moving on to the GUI (web interface, email reporting) pieces of the code. Expect the next update at midnight, EST.

DNS Rejection Lists
In order to cope with the near endless number of misconfigured mail servers (mostly workstations) being used as botnets we have been forced to only accept mail from valid mail servers on the Internet - those with the reverse DNS entries. While we had to throttle it down quite a bit the practice does help a lot in reducing the issues and the amount of float to deal with.

To be clear: We only reject messages from mail servers that do not have a reverse DNS (PTR) entry for the server that the mail is coming from. We do not reject messages for mail servers where the lookup failed (remote DNS server for the zone is down when we run our query), we do not reject messages if the hostname that was returned is invalid, we do not reject messages if the hostname is forged (mail coming from sbs.customer.com with PTR record claiming its adsl-dynamic-63.190.132.22.lameisp.cn).

To be even more clear: This is not a blacklist - this is a network security policy. There is no whitelist, there is no “taking sender off the list” and there is no delay in it at all. As soon as they either start using their ISP’s smarthost as a relay server or get their ISP to issue a PTR record our systems will start relaying their mail.

reject=553 5.3.0 Message rejected. The email you tried to contact does not accept mail from mail servers that are not configured properly, for more details see http://www.exchangedefender.com/help.asp and contact your ISP. Reason: (1) Could not resolve PTR record for x.x.x.x

If you drop to your command prompt and nslookup x.x.x.x you will see why they are blocked, the query will fail.

We send the user to the following page with the clear instructions on what to do so you are not bothered with any troubleshooting of this.
http://www.exchangedefender.com/help.asp

It is worth noting that this practice has been around for years by major ISPs in USA - the fact that there are mail servers out there that still relay mail without DNS (any DNS) means they either just started or something changed in their network infrastructure recently and they are not aware of it yet. It is not your responsibility to fix remote networks.

New admin.exchangedefender.com mail Bridge
Lot, lot, lot’s of issues with the ExchangeDefender daily and intraday reports. We are currently rewriting them, will have it implemented and closely monitored by midnight EST.

In the meantime, we have put in a new network “bridge” between us and the end users mail servers. When the report is generated it will not go through the ExchangeDefender scanning network, it will go directly to the user. This will simplify any troubleshooting and make the reporting more timely and efficient, even though we believe the issues are on this end.

Mail Reports
As mentioned, we are rewriting this piece. Tonight you will get the ability to “resend” the SPAM report (last 24 hours) on demand as the Service Provider, Domain administrator and end user themselves. There will also be a global policy of not sending empty reports to users though we strongly urge you NOT to use it because when people expect reports and see them every day you will receive a phone call the first time that does not happen.

We also recommend loosening the reliance to the SPAM reports. Put a shortcut on the end users desktop to allow them to access the quarantine at anytime if they have questions or are looking for mail. Releasing messages through the web site is far easier and more effective because you can release multiple messages at once. To build an IE shortcut in XP and Vista right click on the desktop, select New, Shortcut and type in this URL

https://admin.exchangedefender.com/login.php?

username=theirusername&password=theirpassword

Replace theirusername and theirpassword with their data and save. They will be logged into your portal and given your branding.

More to follow on this later.

What you can do
======================================

IP Ranges
It is important to have all the IP ranges we relay mail from in the access/relay list. If you do not have us in the relay list, we cannot deliver messages to that server in the timely fashion (messages that fail connection at one DC are streamed to another DC and then delivered that way, adding at least 30 minutes in latency). Please add and use at least the following IP address ranges:

65.99.192.0/24 (255.255.255.0 netmask; 65.99.192.1-255)
65.99.255.0/24 (255.255.255.0 netmask; 65.99.255.1-255)
64.182.140.0/24 (255.255.255.0 netmask; 64.182.140.1-255)
64.182.139.0/24 (255.255.255.0 netmask; 64.182.139.1-255)
64.182.133.0/24 (255.255.255.0 netmask; 64.182.133.1-255)

If you have end user complaints about untimely delivery, unreliable delivery or messages just “vanishing” it is related to the IP ranges not being allowed to properly connect to your server. Either program in the above IP restrictions or do not enforce IP restrictions at all (bad idea).

SPF Records
Configure and enforce your SPF record in your DNS zone. If the remote users are complaining that they are not receiving mail from your users it is likely that the messages are being stripped due to SPAM policies in place. One way to reduce this risk is to define and use an SPF record. You can use the template below and add in your own A records that relay mail on your behalf.

“v=spf1 a mx ip4:65.99.192.0/24 ip4:65.99.255.0/24 ip4:64.182.140.0/24 ip4:64.182.139.0/24 -all”

Finally, I cannot to being to apologize enough for the issues this has caused some of you. All I can do is fix it, as fast as possible, my team is on it and we will not let the weekend start until all the issues are resolved. We have so far resolved the most critical pieces that would cause the end users to contact you (issues of non-delivery, mail vanishing in the cloud, undefined rejections, missing reports, etc). We have closed 60 our of 84 issues I originally mentioned, we aim to reduce that to single figure by the next time you hear from me (midnight)

Please stand by.

-Vlad

The backend at this point works fine. The network works fine. The interface… blows. So we tack that one next. Interface gets completely rewritten from scratch. I start my last minute desperation calls with folks all over the world, trying to replicate the issues, catch the errors, trap them and get the fixes in. I call Mark Taylor from lunch and ask him to help me with one of the issues. He hops onto IM, calls me on my UK number, walks me through the issue.. first glimer of hope, the bug he has had already been closed. On to the fun part.

Friday, August 24th, 2007

At this point days and nights are blurring together. Going into the third all-nighter is not for the squamish, esp when working under stressful conditions in a home office with no more Mountain Dew. Thankfully though, things are rocking and rolling now and everything works.

I spend most of the day in phone calls, load testing and usability analysis. We rename half the stuff to make the interface look and feel a little more consistent, little less confusing. Finally, a week later, the system is back where it ought to be, kicking mail around.

Time for a little dose of humility:

Dear Partners and Clients,

We have addressed all the major issues related to ExchangeDefender that have been present in an isolated fashion across our systems. Everything from service delays, branding issues, missing emails, connections and interface work has been addressed. We have rewritten major sections of the ExchangeDefender frontend and have integrated v3.1 features in more areas than originally planned than expected. The system is pretty much bullet-proof at the moment and everything is working as advertised.

My team has worked tirelessly over the past 3 days to address the issues and settings that have been in the system for years. We have also made changes to make sure this type of an event never occurs again and that we give you more and more control and insight into the system and how it operates.

Everything at ExchangeDefender currently works as advertised. We are taking a little breather today and will resume the development tomorrow and you will receive a new guide to ExchangeDefender v3.1 that explains all the features and how to use them.

I hope that we are fortunate to win back your business. We have learned a lot during the last week and a half and have changed our development methodology to eliminate future “point” releases and events such as the ones leading up to this. We took a risk to rewrite major sections of ExchangeDefender that has been running truly ancient code that I have personally written nearly 8 years ago. I felt it would not be right to ask for you to consider working with us in the future if everything was not perfect.

I believe we are there, I believe the feature set, price, scalability and the partnership support we offer is unique and I hope you consider working with us in the future. You will get a new guide over the weekend, details of all changes will be published at http://www.ownwebnow.com/blog and I just wanted to thank the many of you that have taken the time to help us troubleshoot these issues, get them into our feature and bug portal, work with me directly on replicating the issue and finally getting it all to where it needs to be. I am truly thankful for that.

Again, thank you for your patience and for your business.

-Vlad

Blog a fork in it Jr, we’re done.

Friday Night

After a nice dinner with some friends I get to get this off my chest and figure out just how and why things went so horribly wrong so quickly. It is not that we didn’t test these changes. It is not that we didn’t know the end result of these changes. It is not that we did not appreciate the complexity of what we’re dealing with – this isn’t some monkey banging at the set of checkboxes, these are very complex distributed systems with a very complex management. So where did we fail?

We failed at step #2. When things fail, you roll back. This is where the roles of developer and sysadmin cross in a very bad way. Developer troubleshoots, fixes, and retries. Sysadmin rolls back, triages on a separate system and tries to replicate the problem until it can be eliminated.

Was it an emotional response? Probably. You don’t want people to be unhappy with your product. You want to react quickly and “just make it work” but “just making it work” tends to “just break everything else that touches it”. I call this the reverse-Midas effect, everything we touched turned to shi..

If there is anything I did right in this whole situation it is the communication. Things go wrong. Most people expect things to go wrong. But when they do go wrong you have to admit to them, apologize, and get to fixing right away. Most importantly, everyone else thats being yelled at needs to be informed too. I tried as hard as I could to make sure everyone was up on it. Here is an email I got a few minutes ago:

Vlad and team, I cant express my sincere thanks for taking the issues addressed seriously, you have not turned me away as a client but rather made me a stronger partner in your business. All businesses encounter problems it is how they handle the problems that sets them apart from their peers, in my believe your commitment to resolution and communication is 2nd to none.

My job as a CEO is to shield my people from criticism, motivate them to do better, communicate clearly whats going on with our partners and clients and finally put the resources together to make everyone sucessful. I’m the f’n firewall. It’s not easy working 3 days straight. It’s not easy swallowing your pride and say “Shit, we messed up.” It’s not easy watching things fall apart, one after another, on something you have built from scratch. It’s not easy, as Dana Epp says: if it were easy everyone would be doing it.

So yes, this was the worst professional week of my life… our product and job is not easy, but, I cannot imagine doing anything else. To my team, to ExchangeDefender, to Microsoft developers that get the third degree like this every day, to all of our partners and customers that stuck with us and helped us all get to this Friday: This Bud’s for you.

Matt just called to ask if I had anything special planned for my 1,000th post. Huh? Apparently, this is post #1,000 on vladville.com:

I had to include the SPAM count as well, pretty impressive. I also like the number of comments - for everything I say there are at least 3 people that say something too. That’s precisely the point, to write (talk) about issues and air them out.

Over the past two years you’ve seen me experiment with a number of community initiatives (some successful, some spectacularly embarrasing) business ideas, opinions, technology articles, roadmap thoughts about Microsoft/Linux/Apple as well as everything (and I mean eeeeeeeeeeeverything) that was on my mind. If there is one thing you can say about me is that I do not hold back. But, I have learned what works and what doesn’t and I think it’s time to start with a new plate.

Now what to do, what to do. This is something I’ve actually been thinking about for over a year when David Overton asked me to clarify if I still felt the same way about some bug that had been closed months ago. So, what am I going to do between posts 1,000 and 1,001?

I’m going to delete them. Everything aside from Vladfire, VladCast and SBS Show is going bye-bye. If there is a particular Gaypile post or flamedown that you’ve had your heart set on its time to snag it now because by the next time you see Vladville it will be gone.

Update: Wooops. Nuked the last 10 posts before I realized that I might want to have a backup for archival purposes. Anyone have a plugin that can archive posts to HTML for Wordpress (yes, I could easilly write it myself but the plate is kind of full at the moment)

-Vlad

This weekend I will be chilling on the beach activating some 500 or so accounts under Shockey Monkey. I have not done it in about three weeks and well.. they piled up. But I have to commend the folks that signed up, everyone has been totally respectful and waited (and put up with me) so you’ll soon have your own monkey.

The delay is not just due to my incompetence but also due to the huge software launch in ExchangeDefender v3.1, in the development work on Thieving Weasel as well as a more rapid development schedule and need to be “there” soon. Looking at Shockey Monkey its easy to tell it’s not “there” yet but as I keep on getting folks that overpaid for other solutions, the ideas and demands are going up as well. I am doing all I can to justify the faith and the switch folks, believe me, working my butt off on this.

I would also like to thank all of you that support Own Web Now in this effort by sending us your business. I really, totally, and absolutely appreciate that. Having OWN make money means more money for monkeys means more time for Vlad to put out an affordable solution that is as inexpensive to implement as possible.

As always, thanks to all the partners that signed up for SM that continue to file bugs and suggestions so that this show goes on and everyone wins. I just want you to know that I appreciate it.

Just 7 days left till the launch….

Badass, ain’t he?

Ironic that I would say that out loud but people have just been killing me all this week with their Out of Office responses. Particularly UK people. Here are a few samples.

I will be out of office till September, please contact xyz.

I am on my annual vacation from Jul 20 – Aug 20th.

A month long vacation? Now this hurts particularly bad given that I am here on a Friday working like a dog because I took Wednesday off to go to Miami. How come we don’t have it this good in USA? Aren’t we supposed to be the worlds laziest nation right after islands in the carribean?

The title in itself is my thesis. It is something that has been on my mind for years, but was prompted by this post by David Overton. I am of course not giving away all the details and secrets, as thats what you go to WWPC for and what defines our business value, but you can get the general idea of how we have been running and intend to continue growing our business in the future.

Last month Microsoft’s chief Kevin Turner introduced the Software + Service initiative. Allow me to sum up the strategy for you: “We hope to be where Google is today, in about two years”. Thats all there is to it folks.

The inherent evil in this strategy comes from Microsoft’s almost decade-bred megalomanic plan first coined Hailstorm that has over the years advanced through MSN, Live.com, Windows Presentation Foundation, Silverlight and now S+S. Again, for the sense of simplicity, Microsoft wants licensing (read: control) over every technology implementation used by computers and humans. They not only want the royalties but also control and direction management. They just need some stupid enough to sell it, it is the only missing piece.

Understanding Microsoft PM’s

Microsoft Product Managers are the folks that “own” the code and “own” the direction the product takes. These are usually quite brilliant people, with lots of dreams and ideas, like almost all other developers and strategists. When you have a dream, it is hard to let go of it. It is hard to go a day without thinking about it. So you try – and you fail. And you try again, and you try again, and you try again. You keep on trying until someone up high recognizes the potential, throws money at it and decides you’re going to be the next star.

Take a look at this blog post that sums up how software goes from concept to delivery and how many obstacles Microsoft PMs face.

Understanding how Microsoft is not Evil

Many (Linux / Apple / Mozilla) zealots hate Microsoft and often assign unfair criticisms to it. One of the local Linux guys is often saying, incorrectly, that Microsft is a “felon” that has been “convicted” many times, all of which is false, but in the same breath talks about how much he loves Google and Gmail and company that claims it is up to no Evil.

In my career, I’ve met Microsoft employees of all levels, from CEO on down to the guy that packaged the software on the Windows 1.0 assembly line, and I have yet to meet an “evil” person. However, Microsoft is not a company of employees, it is a publicly traded corporation with shareholders, stakeholders and greedy investors demanding more. Microsoft is also a corporation that has had more fines assessed against it for patent violations that nobody seems to mention in their articles where they try to guess how Microsoft will try to exert control over the entire world with their new “bs” patent.

So Microsoft, in its nature, is not any more or less evil than Google, they are after the same goal. They want the entire world to use their platform, their software, their solutions. They want to be the defacto . . . something. 

Understanding how Microsoft is Evil

Microsoft is evil in its business practices. There are tons and tons of articles you can read in which you see Microsoft destroying entire business verticals through poor software, key acquisitions, dumping the product for free to kill off competition and then either letting it die or making it very commercial.

Microsoft is evil in seeding developers on their platform. If you start a software company, Microsoft wants to be your best friend. They will send you tshirts, free software, free phones, free development tools, free training, free anything. Just for the love of god don’t look at the other, open, solutions because Microsoft’s closed ones are more reliable and secure. Besides, they own 90% of the market, how can you go wrong selling to that large of a crowd?

You go wrong by spending a year or two bringing your product to the market, just to see Microsoft cherry pick the solution and make it the part of the next release. Or they buy a competitor of yours, bring in the feature in-house and then by marking it a part of the product try to reach out to all of your customers and tell them to do it. They also have the balls to come to you, tell you to stop your development, and instead try to make a business out of implementing your former competitors solution and be an “integrator solution shop” – I am not joking.

Understanding how Microsoft and Google will kill the Small Business Specialist

SPFs/OMBs/Generalists are going to die.

Quickly.

I have said this in public forums before and have gotten attacked from every angle for sharing my knowledge and experience with folks. But allow me to explain what I know – as a mere CEO of a company with the largest cloud deployment of SBS and all its friends (Exchange, SharePoint, etc)

Cloud is sweet. It absolutely is. There is no way to fight with the cloud and win. Why? The cloud is cheap, its a low monthly fee that is predictable. The cloud is scalable, as a simple variable increase in the cost of the monthly fee you pay (Exchange Standard hosting = $10, Exchange Enterprise hosting = $15; Try doing that with a standalone on-premise server and it will cost you thousands and thousands of dollars in reprovisioning and infrastructure upgrades). Cloud, for the most part is more reliable than the onsite deployment, I also base that on our internal knowledge of the sites we manage in our data centers and our customers networks. Cloud is always evolving, always migrating – there is no defered upgrade, there is no migration strategy, there is no competitive exploration, there is no security software option – its just a monthly fee and one day you’ll wake up and the new software with new features will be staring back at you.

If you are an SBSC, your chances of pulling a customer from a cloud based monthly fee to a local, single point of failure, prohibitively unjustifiable infrastructure investment with hard limits for a company that may have a growth or consolidation plan are slim to none. Basically, forget about being a small business specialist in the cloud based environment.

No really, they are going to kill us.

Microsoft (and Google) spend a fair amount of time trying to assure us that there will be opportunities, that we can keep on consulting, that we can make money…

Truth is, they are after a referal at which point they can take over or reassign it to the lowest paid employee in the company. Take a look at Windows Office Live Basic hosting which is free. If you are a web hoster, developer, web designer, etc, etc – you are soon to be out of a job. Unless you are stupid enough to think that a small business will pay you $100 or more an hour to sit there and fill out web forms, be prepared to just advise the client where to go for their web site and enjoy your walk back to your car while you can still afford it.

That is very cruel but I did it for a reason, I wanted you to feel the contrast between the worst case scenario and best case scenario Microsoft and Google are pitching.

They aren’t liars (any more so than the average marketing folks) but they are there to kill us. Microsoft’s longest running line when talking to partners has been:

We have a solution you need to be excited about and sell to your clients.

Sell, sell, sell, sell.

Microsoft doesn’t sell software, our partners do!

My expense reports tend to differ with that, but I digress. S+S means Microsoft will be selling to the customer. Directly. As in completely bypassing you. You will be sold on the opportunity of developing some really high end stuff that most small business shops will never afford, but Microsoft will try to hit up the customer directly and hook their SBS box into their solution. Look at Exchange, you’d almost think its impossible to have an Exchange deployment without the Exchange Hosted Services, which are really neither but again, I digress.

Microsoft is actively looking for direct contact and only looks at you as the sales person. I hope you get comfortable with that, because Google, which is working on the same strategy, has no place for you at all.

Where they will fail…

Microsoft can’t sell to the consumer.

Microsoft sells to techies. Over 30 years of selling software to technology people, enticing technology people to talk to the business people and consumers and try to sell them on the latest Microsoft innovation. But when you look really, really hard, you find a sea of Microsoft failures at best (Zune), and open cash hemmorages at worst (Xbox) with pretty much a graveyard of flashy marketing in the middle.

But Microsoft will fail.

What Microsoft, it’s sales people, its developers, its PMs and others do not understand is that it should NEVER be called SaaS. Pointy haired bosses seem to say its all about the name. And they are right.

It’s saaS, bitches. Service is king.

If you want to see the SaaS failures, take a look at this post written by my dear friend in her time of panic. There is no service in the cloud. There isn’t even a phone number. Your intelligence, your data, your business.. can be wiped out as effortlessly as it was setup and you are just expected to live with it.

Until Microsoft, and Google, realize that it is all about SERVICE, they will continue to pile up large losses, customer disinterest, consumer freeloading and marginal success at best. Why? Because although customers buy the software, they buy it for a benefit. They buy the service. In Microsoft terms, they buy the Assurance in Software Assurance, assurance that the next release will be free. And when that doesn’t live up to its promise, customers end up pissed off.

So Mr. Gates… Steve… Personally I like you, professionally I wish you the worst of luck and an assured failure because you simply do not get it. Service is a personal connection, it is not just service because we say it is because we crossed off a bullet point on the requirements checklist and hired an indianinabucket.com to support it.

Service is care. And you are setting out to destroy, financially, the very partners that could help you with it.

Written by a CEO of a profitable saaS company. And thats about all the advice you get for free.

Now, if you aren’t pissed off at this (be it Microsoft, Google, SBSC or Steve Ballmer… you haven’t learned a damn thing. Read it again).

This is a guest post by Vlad’s wife, Katie.  Not only do I actually exist but I am in need of help.

As Vlad previously mentioned, his birthday is quickly approaching.  My attempts at prodding for present ideas have proven fruitless so I now turn to his very own blogging audience.  What would you get the geek that has everything or openly mocks the products that he doesn’t already own?  Is there anything on your personal wishlist that you could recommend?  I am down to ten days and desperate.  Any suggestions are welcome!!

Items already vetoed include the iPhone, Wii, and a steering wheel for xbox 360.