How I became the “Grid guy”

Hi,

In one of my last posts, I quickly explained I worked in the
SharePoint Engineering team for few months. The experience was amazing, though
pretty challenging. I found out a lot of things on a lot of topics, but I loved
meeting smart, dedicated and great people.

Some people wonder what I was doing there.

 

Here are some clues in
this post…

 

You may did read this post few months ago from Jeff Teper
(SharePoint Corp VP) https://blogs.msdn.com/b/sharepoint/archive/2009/10/11/engineering-sharepoint.aspx
. It was a pre-SharePoint conference post where some of the engineering
organization details where explained.

In this post, you may notice this section:

“Grid
Team – In the last couple of posts I mentioned that SharePoint Online is big
part of our development cycle. […]. This release, we created a dedicated team
called “The Grid” team to analyze and optimize in great detail the total costs,
time, reliability and other factors of a SharePoint deployment we hope to reach
10s and eventually 100s of millions of users. Almost all of what we learn there
helps make the product more reliable for customers and partners hosting their
own servers from a small company to a 250,000 employee enterprise. There is a
bunch of great learning from things like virtualization with Hyper-V to best
practice analysis to operations monitoring we will share in the coming months.
The Grid team will be at the SharePoint Conference.”

 

For those of you who attended the SharePoint Conference, you
didn’t see a “Grid” session or a “Grid team” at asks the experts. In fact they
were here, but with their “classic” SharePoint titles.

In the excerpt, you read “we will share in the coming months”.....

 

Well, here you go: that’s partly what I was requested to do.

• Why me?
  • Because of my work on SharePoint Automation
    (White Paper, TechReady internal sessions, automation tools and scripts, blog
    post series on SharePoint developer workstation automation).

• Why
  SharePoint Automation is important?
  • Because it’s the key to reliable SharePoint
    platforms at any scale.

• What does
  it mean?
  • It means SharePoint Engineering is taking the
    effort to develop SharePoint Online (Standard). In fact that’s the directions
    from “we’re all in” for Microsoft: Products need to be built for On-premises
    AND Online deployments…. And in between (mixed, hybrid, whatever you want to
    name it).

• What was
  done?
  • With one of my colleague, Senior PFE from MSFT
    US, we interned the Grid team (I mean, being part of the team for months) to do
    various things:
    • Understand how Grid team plan to do SharePoint
      Online (I say plan because the General Availability is beginning of 2011 now –
      so it’s not here yet)
    • Abstract the concepts and tools used to fuel
      SharePoint Online 2010
    • Start internal evangelism on these concepts, in
      conjunction with the Dynamic Data Center team
    • Work on mapping these concepts and tools in
      Microsoft SKUs
    • Build awareness in SharePoint internal community
      on this “way” to fuel an Online Service based on SharePoint

So, step by step, that’s how I
became internally the “Grid guy”. With my PFE colleague we led few webcasts and
technical internal sessions/events to start the messaging about SharePoint
Grid. Being passionate about those topics of Automation and Orchestration,
people came to me on these aspects.

We also presented these concepts
to various internal stakeholders, and few customers interested in hosting their
own “Private Clouds” with SharePoint being part of it.

To be clear, “Grid” is a
SharePoint Engineering set of concepts, tools and processes. They own this
thing, and are quite amazing on how they do that. But out of the Building 16, I
became the “Grid guy” – its “public face” like the Grid PM lead said once – of
this work. Just because I was asked to understand, and explain outside of the
building how it’s done.

To test my understanding, I
reproduced SharePoint Grid on servers in France. That was pretty good
experience to do that. I needed to check what it was and how it worked. That’s
the way I am, I like to know what I talk about, up to be able to do it myself
if needed J.

• What was
  delivered?
  • Internally, I delivered a lot of documentation,
    workflows, mapping, Technical References content plus discussions with different
    Product Groups. It’s all available -internally. With the rest of the involved
    team, we also developed an end-to-end approach to think workload first, then
    sourcing options depending on the use cases.
  • The team received the “CTO collaboration award”
    for this.
  • With one of the initiative I worked on (called
    “Image Factory” – to come in another post), I received the “Technical
    Excellence recognition” from the IW community (kind of internal MVP thing).

• Now what?
  • Grid team has been reshuffled with Office 15
    reorganization. The small and highly motivated team that was the Grid one became
    SharePoint Core Data Center team. They now have much more resources to ship the
    product Online. Things change. What MSFT learned from this Internship is that
    the Learning curve really starts after the GA, when customers onboard…. So,
    that was too early to “share in the
    coming months” how SharePoint Online is done. We’ll probably do that after
    the GA.

• What SharePoint
  Grid means?
  • GRID itself means nothing. It’s not the acronym
    of something.
  • The project needed a name and the SharePoint
    Cloud for MSFT is SharePoint Online. , so another name was needed for its
    underlying infrastructure.
  • But I can share something pretty simple and
    clear on what SharePoint Grid is:
    • Imagine numerous Hyper-V hosts, with the exact
      same hardware. We call it a “compute resource pool”.
    • Then implement Virtual Machines which are “Roles
      based”: in the case of SharePoint, we have 3 obvious roles: SQL server,
      Application Server, WFE servers.
    • Of course, you can add more roles and diversity,
      like dedicating servers for Sandboxed solutions or Office Web Applications.
    • What do you end up with?

A
“SharePoint Grid” you can represent like this:

	Host 1	Host 2	Host 3	Host 4	Host 5	Host 6	=> etc. =>
VM1	WFE	APP	APP	WFE	SQL	WFE	…
VM2	WFE	APP	APP	WFE	SQL	APP	…
VM3	SQL	APP	WFE	WFE	APP	SQL	…
VM4	SQL	WFE	SQL	SQL	WFE	WFE	…
VM5	SQL	SQL	SQL	SQL	APP	APP	…
VM6	SQL	WFE	WFE	APP	WFE	SQL	…

 

Nice, isn’t it? And that’s just for 6 Hyper-V hosts, hosting
each 6 VMs….. But it’s already 48 Windows Servers to operate and manage.

How can you do that? ….. Automation with new concepts is the
only way J …
and that’s my passion!

 

I’ll share few useful insights and background in the weeks
to come. First one to start:

• The Building 16 lab used initially to develop
  Grid, had to manage and pilot more than 900 servers (physical and virtual)…..
  1/3 of them where rebuilt automatically every week!

< Emmanuel />