[EdLUG] Advice on Linux migration task

[Callum Scott]

Im a bit late to the party here, but seeing as I am currently in the midst of a migration from on premises services into AWS I figure I can at least provide some pointers to expand on Johns point.

The work I am doing is largely non transformative, that is to say that we are not taking services and splitting them up to be AWS native.  However, we are utilising AWS tooling where it makes sense to do so.

First thing is not to think of the numbers of servers you are tasked to migrate.  You have a number of services that you need to migrate, which will be made up of a number of servers.  Your goal should be to migrate the service, not the server.  Lifting and shifting servers into AWS is a great way to spend loads of cash and pile on more technical debt to your exiting debt without addressing any already existing.

My current task, that I am just code complete on this week was to migrate a server that ran about 20 odd bash scripts in cron jobs.  Rather than lift and shift we decided that it would be better to build out a terraform module to allow is to manage scheduled tasks in Fargate and Event Bridge.  This way we get our cron jobs, they only cost us when they’re actually running and we also get the option of introduction event driven “cron” tasks later.  This is all very nice and shiny.  I started that around May and we are only now in the endgame of testing.

Why did it take so long?

1) Discovery took ALOT longer than initially expected 
2) We didn’t have a method of executing scheduled tasks in Fargate and Event Bridge, so that had to be built.
3) We are reliant on some 3rd parties to do stuff like firewall changes and sending us files that the cron jobs consume.

By far the most difficult part of this task has been the discovery phase.  Of the initial 5 days slated for discovery I have probably used at least double that if not more on just understanding what the system currently runs, what it interacts with, who owns which cron job, if that cron job is even required anymore etc.  

Sure its taken several times the timescale you have, but from a business point of view its a win:
	- From the original 20 odd tasks I am migrating half a dozen.
	- We now have a scalable and simple solution for others to add scheduled tasks with 10 lines of terraform in their module
        - The solution costs practically nothing to run.


You are being asked to eat an elephant, and if you just start biting chunks you’re going to throw up.  If I were you I would tackle your problem in roughly the following way.

Group the servers into services and identify the stake holders.
Get a broad idea of what the service does (is it a mail server? is it a web service? is it a proton accelerator?)
Perform an initial discovery by logging into the server(s) and poking around.  Document how you think the service works.
Interview the stakeholders and make sure that your understanding is correct and what the business impact of the server is.

At this point you should have enough information to architect the AWS Solution and a migration plan.  Only at this point will you be in a position to estimate how much time the actual migration work will take.

For a single or dual server service that is about 3-5 days work.  For anything bigger add a day per server as a finger in the air.  (Even those estimates assume the service is fairly simple and all managed by Ansible or Puppet or something sensible).

Some simple maths to round off, assuming every server is an isolated service:
10 weeks is 50 days.  
40 servers @ 3 days per server for discovery is 120 days.
So you are already at 24 working weeks of discovery in the best case.

You need to have a sit down with your PM and come up with some realistic expectations for scoping out the work first.

Hope the wall of text helps in some way.

Callum

> On 19 Jul 2021, at 20:40, John Dow <jmd at nelefa.org> wrote:
> 
> 
> 
>> On 19 Jul 2021, at 19:33, Martin Love <martin.t.love at gmail.com> wrote:
>> 
>> 
>> Hi Tahir,
>> 
>> The first think I would be doing is setting expectations with the Project Manager.
>> 
> This, above all else. A project like this needs to be clearly defined if it’s to succeed, and that’s the project manager’s job, not yours. 
> 
> The timescales you mentioned are completely unrealistic for essentially reimplementing the services provided by the machines.
> 
> Additionally, the “seems to” in your question suggests that you’re not 100% clear on what it is they actually want. When you say “migrate from one organisation to another organisation”, what do you mean? 
> 
> John
> -- 
> EdLUG mailing list
> EdLUG at mailman.lug.org.uk
> https://lists.edlug.org.uk/mailman/listinfo/edlug

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.edlug.org.uk/pipermail/edlug/attachments/20210721/c91cb70d/attachment-0003.html>