back in India, I started this ascension with the multinational company. I worked there for two years, and during the course off, the time I realized that, uh, the domain knowledge I had their expertise I had with all the different foods that offers are the technology stack that is needed to progress was lacking. I didn't have that, Uh, just after my undergrad. That's what I felt on the reason and blow up that I thought of doing that up, scaling myself and doing my master's at the University of Utah Eso I moved to America here on and, uh, Masterson information systems with a focus in Data engineer. Um, although I will assistance engineer. But I always had a thing that I was not a good programmer. I'm still not a good programmer on duh. The reason I don't know what the reason for that is, but I just knew it deep down inside that I would never be able to court the bay people too. Um, and that's why I was focused. My career part words data engineering and also a little bit off develops kind off rules, So yeah, So I decided to come and do my information systems matters and pension systems. Utah United of Utah, on. During that time, I did an internship with Indus Valley Partners. So it's a software company in that helps hedge funds to manage the date of their house and help them report on a daily basis on their current positions and stuff. Um, while doing the internship, I realized, uh, that because I was distance engineer. A lot of things that I required to be a engineer. Um, Waas I didn't have the, uh, that experience and I was doing the internship. Got that experience off working. We did our houses. And how did the ingestion works? How they have built the pipeline so that they have reporting done on timely basis. Uh, because it's very important in a company in our industry. Ver ah. Markets are so volatile on if report is not done properly for that day can impact a plant. So, um, I got the experience of working in a very high, um, highly, uh, what to say, uh, an environment where it's a lot of pressure to deliver. But at the same time, I learned a lot off stuff that can make your life easier. Ah, especially the data intimating and did about housing. Uh, but when I was doing the internship, I did take up a full time job there because I like that when I was working on. But after working for like for some time there, I realized again the stack that they were using waas a little older and people were moving towards using cloud technologies in order to make things easier. And a company like this, they should be really focusing on that. I was not seeing that for a class for next 23 years. They're focused getting there. That's the reason I guess I could switch my job on and I joined Fidelity Investments. So currently I'm working. There is a clouded engineer, and, uh, I really love it. That because one of the reasons I love it there because not just because they're not using a legacy like it's a system they are using, like a system, the whole databases and Oracle. Ah ah. But they are moving towards all this latest technologies that I require toe be keep up with their competitors, and that is giving a party to learn the the transition from, um older technologies to the new one, and it has to understand how they actually help organizations. And so, yeah, right now I work. It's oclock it. Engineer ability, investments. And I love it there. Ah, alone, new things. Every sprint on that keeps me going with that. With that with that company never makes my job monotonous. And that's what is required, I guess, on the job.

it's a 40 are weak. Ah, a typical 9 to 5 job on Vegans are off. Um, I started with a data migration project, which Ah, So, as I said, they had a legacy Oracle system and they wanted to move to cloud. They wanted to use the AWS idea services. Uh, but not just that. They didn't want to just, ah, do ah, lift and shift kindof migration. But they wanted at the thing to be streamed, um, slowly side by side in a panel manner where in the article, databases would still be found so on they slowly migrate all the data different other services based on the knee and use off that data are the consumption of the data, uh, and then slowly migrate. There are legal system. So it's a two or three years off project that in some of the data is getting fed to an elasticsearch for real time reporting, but that we're using Landers and Knesset streams directly stream the data based on real time events that are coming in Oracle. Um, that's one of the pipelines that walked on to have ah ah have that Guinness extreme set up, uh, and also Lambda us that would actually feed off off the geneticist E to push data to s three and then as the and elasticsearch. So the S three push was for some. That was for some other use case. So that's one of the pipelines I walked on. Um, then currently, I waas uh, I I'm a part of a team really trying to, ah, build Python C allies that would allow different teams to deployed Davis Custer's in AWS. So Fidelity Execute Organization. They have a lot off governance and business specific requirements that the teams, if they if it is given the teams, it becomes difficult for them to manage. So we want to extract all that and just give them ability to focus on business functions. So we're writing pythons realize that would allow them to just use a c l. A. And deploy that pastors and all the governance off the cluster Ah, the data longing and all that stuff is handled by the script at the back. So that's the current thing. I'm working out

Yeah. So for data streaming the majorly use Guinness's data streams for real landed at streaming, uh, for archiving data. We use AWS s three to manage all the transaction data. Really use it in this ideas. And based on the requirement, you can either use post dress or my askew. Ah, favor. Off the off. The A team is on the assistant idea service. Ah, for deployments. We use Jenkins, and I think now there's all them a data engineer. Uh, it has become imperative folk for everyone to learn one deployment management tool. So we're using Jenkins for that. So it helps. It actually manages our deployment pipelines and help us scheduled deployments properly. That control languages. I talked with Java typescript and python. I started the lander that I wrote for data streaming more than typescript. Andi, The CIA lives that were writing right now is in python. Also for data streaming. There was one processor that actually query data from Oracle. Um all right, pushed it into the kinetic string. So that processor which ran on independent, easy to server, Why is it in Java? So there are a couple of things that have I would not say I was the only guy who rode all this stuff. But when it comes to having exporter de Villa three languages I had exporter whip on and inside AWS there were even Mawr services other than s three easy to look in. Essence slammed us and ideas that we used to do. The song stinks in our tradition, Yeah.a Z. I was talking about having the knowledge of deployments along with the, uh with the way the Creator is right now, which is moving towards more off devolved kind of culture. Snow artist. It's very important toe. Have the knowledge off deployments along with the pipeline for the dinosaurs Fight night for suffering this applications so it can be different things. But if you're not able to actually deploy it out, it's off. Not much use. So if I had to suggest some of the deployment pipelines are tools that people should focus on, would be one will be Jenkins that I'm personally working on. So it has its own. You I It's nothing but an orchestration off your deployments. Very you can see better. You can actually tell that software forced to build your application. Test it out, see if the tests are fine and not, and then you can actually deploy stuff up. So if you had to do it yourself, you would actually have to manually do it like first build it and then pick our ticket out. And so they never thinks I don't manually. It's very easy to do editors and in a production environment. When they're working for a company, you can't do that. So having an idea off tools like this is very important. And I think most of the companies use Jenkins. And the more important is not learning the tool as such, but to learn the concept of deployments. So whenever the application is there, uh, you want to take it all the way to production so that customers can use it Now you, for example, you build up their friend work. You have about application, you have some kind of framework, maybe angular, our spring. But whatever it is, once you do that you have done it on a local machine. Now you want to host it somewhere on some server on. You also want to run tests and also wonder maintain a repository that would allow you to make changes to that application. Right? So first thing is having a repository cancer deployment. So most of the people use backpack get basically, that's what they use. All the core base is there in get, and that's the first step. So you do all your stuff and get have the code base application coded. There, you do it from a local machine push it to get extended anymore. Prancer the next step is actually testing it out in an automated way. So you would write on the unit tests, integration, test our function tests that are there. But you want to run them, uh, on a server on a daily basis, like for before each comet. You do you want to run a test to see to it that existing changes are not broken. So once you do that, but you have a get repository. All that can be automated using junkets so you can write all your scripts to actually run the tests. Uh, see, what changes had their Andi bill your project. Every time a new feature is added to your project, so four stages accurate building and testing in Jenkins. Now I'm sending about Jenkins. Now you can do that using a different software also, but the build intestate for the doctor application is going to remain for all the technology on what Obama's offer use, then the next thing is, once you know, your testing is properly done, you want to deploy that to ah non production environment first to do some kind of integration test with a different application. Your service may be using someone else's service, so that's the next stage. But in you acted glide toe a non for environment, and then you test it out again, and the last day it is deploying toe a production environment where your customer uses it. Now, if you talk about a Springwood application, you want to build it. You may be using maven upgraded to actually build the project. You build it out and then deployed on. Let's say if you talk about, um, deploying your application, it has to deploy to some mushy right? It can be an easy to container. It can be a docker container. It can be on a kubernetes cluster. So all these technologies, like E. K s or Easy to our doctor, these are nothing, but, ah, the So if you have to run this in a local machine, you would just type of the commands. And if you run a local machine now, then you run and get on a server on order on independent. You did some kind off hardware. It's not. You got to see it, but you need some kind off computing power to actually host your cord. Right? So all this technology is not there. You can use them like if you want to. If you have a micro service architecture, you use docker. Uh, if you just. If you have a standard application, you will use easy to containers. Now, if you have applications that are dependent on each other, you use Cuban disc Lester because you can actually orchestrate different components of her application in just one refinement. So based on the requirement off your application, you would actually use a particular technology. So understanding the underlying architecture er rather than just learning E t s are just letting doctor or just easy to is more important. That's what I feel personally, to understand how that whole process works off deployment. And why do you have to use one of that? The technology is more important because if you don't understand docker, the concepts behind Docker, what it does, you would not understand when to use in the city continued or in a container. So so, yeah, Once you do that, you can deploy it in one of the one of these containers are servers. And then it was hosted for the customer to use. Now, if you're using Jenkins, all this is automated because you're writing scripts. Do that. Now. This krypton can be done using python or baskets, so the the other other two languages, which are like a must other than any of the best frameworks, are application specific, uh, programming languages that you have to build applications. You also need to learn a scripting language because all this orchestration is going to be done either using python or basket thing you're not going to use Java are a typescript to actually do that because it's a heavy land. So you lose some light with language that you can do all this orchestration. So, yeah, and it is a very white a pie in any role nowadays to also have a knowledge about on the deployments, along with medication everright. So, Lennox, self stripping is they're in everything. Like I actually was first, given a Windows machine and using limits on windows is a hassle. So I actually had to tell them to give me a Mac, because are actually purely next machine. You can I would advise any of the any students out there instead of using a Windows machine by a Lennox machine and work with it because the commands that I used in orchestrating all this stuff, they are actually the same commands that you would run on your local machine. And all this is mostly than is Lena in Lennox, you don't use Windows operating system or Mac because you want something that is raw in its nature. Um, and you can do all command lines, um, orchestration off this stuff. So you want to do everything that is available? Why? Command line? And I think there is nothing better than having a Lennox operating system, um, on the server. So when I said when the deploying things on the server, it has to have operating system to it. And that is Lennox. Basically, it can be of any flavor. Can be red hat are a boon to but at the basis Lennox that is actually hosting your application on orchestrating all that stuff inside the container. So very important to have Lennox knowledge and on the best thing is you can't learn it in one or two days. So it's best to start using a Linux machine to do all your stuff. And that would make you very custom to all the commands that are using. And you don't have to actually remember them. It was just to just get to you when you're using it.