Sohan's Blog

Things I'm Learning

Context

Misfit

Image credits to NCinDC

Context is everything. Due to a context mismatch projects such as exporting democracy, agile transformations, building startups, losing weight, mindfulness or anything you can think of fail to repeat.

Context is formed by why / what / who / when / where / how, each of these being a critcial part. Most often it’s common to focus on the how alone - forgetting the rest of the context while adopting a recipe for success in business. Some get overly influenced by speakers and focus on only a subset of the context. Reality is, it’s almost impossible to find an excact duplicate of any one context. So, while copying solutions from one context, we must adapt to account for the uniqueness of our own context.

A lot of innovation lies in the context gap - where one can find clever ways to adapt solutions from one context into another. I’ll give you one such example, did you know Uber runs a motorcycle based rideshare in many countries?

Change

Manager’s Mind

As managers, we have the benefit of view because we have our eyes on longer term goals. To achieve those goals, we often have to introduce a change. For example, we may want to change how a team is formed or what product a team works on based on new business needs or opportunities. So, we put a lot of thought into it and propose our plan to the team hoping the following reaction:

1
$(team).on("change", excitement)

Before proposing such a change, we’ve analyzed various options, including the option of keeping everything as-is. We’re fully convinced that indeed the change we’re going to propose is the best option. We are excited with this new way that’ll make our teams more productive. As caring managers, we are thrilled because we believe this will make the team happier.

Team’s Mind

However, most of the team’s reaction is as follows:

1
$(team).on("change", panic)

Honestly, the team has a great chemistry going on, they have fun and work well together, they are already so productive shipping products using technology they use today. So, at best a change seems random, but more commonly perceived as a mindless act of “management BS”.

The Deadlock

At this stage, both the manager and the team are in a deadlock, each party fairly confident in their judgement, failing to appreciate the concerns of the other. At this point, it’s the manager’s job to not only break the deadlock but also to make sure s/he carefully listens to the team and motivates them. Ideally, we can data-drive these conversations while being mindful of individual emotions.

The Checklist

Here’s a checklist I’m following myself as a manager as I’m trying to orient my teams from technology driven teams to more full-stack product mission oriented teams:

  1. Motivate the team with a why. e.g. Go from tech-oriented teams to product mission teams so that all of you can deliver products to customers…
  2. Listen to the team. Multiple one on one sessions if needed. e.g. Let’s discuss about your concerns regarding tech debt with this change…
  3. Discuss changes to your plan based on the team’s feedback. e.g. You’ll have full ownership of the technology to serve your mission…
  4. Layout your evidence. e.g. We built an improved search feature that we couldn’t ship to customers for months because the UI team was busy…
  5. Discuss pros and cons of alternatives you thought about. e.g. Another option is to keep as-is, but that’d mean we must improve cross-team project management to reduce hand-off related delays…
  6. Present your metrics to achieve. e.g. Time from dev to customer, performance metrics,…
  7. Present an iterative and incremental plan with the option to change course based on metrics. e.g. Let’s do this with one team for one month…

When I Applied KonMari to My Calendar

Marie Kondo

“I’d like to tidy up the entire planet. I would go anywhere if there were something that needs tidying.” - Marie Kondo

Well, she came to my work calendar in spirit. Let’s start with some backstory.

When I was developer, my typical day looked like this: one standup meeting and then pretty much the rest of the day in coding. Sometimes I paired up or jumped on a whiteboard session with my coworkers to hash things out. Apart from this, we had bi-weekly meetings for demo, retrospective and planning. All in all, less than 3 hours of meeting on a 40 hour work week.

I’ll admit, the 3 hours of meeting each week felt like a “waste of time”.

In my current role, I’m leading a total of 8 small teams organized as follows: 1. Web (6) - Production engineering, UI/UX, identity and access management, and 3 feature teams 2. Data (2) - Stream and search teams

I’m very blessed to have capable leaders for these teams. To ensure I support my leaders and provide the teams with necessarily context, my typical days in my current role are very different from my developer days. Honestly, when we grew too fast, I was feeling lost in so many meetings. It felt like my job turned into just going to meetings and even if I had an open slot, I was often too mentally overwhelmed to make a good use of it.

Thanks to the mindfulness training that Cisco offered to our whole group. It inspired me to rethink my work hours. After a few trial attempts, I have now finally settled into a routine.

First, I have reduced my meeting obligations. I only attend meetings where I need to take part in decision making. I follow the Amazon way of producing a written document if I host the meeting. This applies to my one:one meetings as well.

Second, I have shuffled my week such that I can have each day of the week organized around a theme. It’s not perfect, but for each day of the week, I clearly know the main theme of the day. It transformed my mental state from “feeling lost” into “looking forward” to the day. It also reduced the wild levels of context “swing”.

Today, here’s how my weeks look like:

  1. Monday is planning day - web team.
  2. Tuesday is product day - to work on engineering problems related to our product roadmap.
  3. Wednesday is people day - one:one.
  4. Thursday is maker day - hands-on R&D.
  5. Friday is planning day - data team.

Because our teams plan for two-week cycles, I have a decent amount of free spots on Monday and Friday every other week. This allows me to help the team, mostly with pair programming or discussing potential solutions to imminent problems. Also, it frees up space for ad-hoc meetings and hiring activities.

The transition from a developer to a manager and then to a manager of managers can be jarring for the “makers” among us. I hope shaping your week in themes like the ones I shared here will help calm things down for you.

Happy leading.

People Management and Diagnosing Low Performance

Beginning with a story here. Back in 2008, I was a couple of years out of college and working for a software company in Dhaka, Bangladesh. Like many of my classmates from my college, I was studying GRE alongside my job for a PhD admission in a reputable US university. That process being long and uncertain, I also applied for an MBA program at the University of Dhaka as a backup plan. The GRE preparation came in very handy and I qualified for the evening MBA program. I started MBA classes in January 2009 before the universities in USA and Canada made their decision on my application. There, I took a course on Management. I was too naive and still too much an engineer to pay attention.

But this image that our teacher drew on the board got stuck in my head to this day.

Performance

Fast forward ten years and I still like how cleanly it captures the essence of high performance of an employee as a combination of the employee’s ability and motivation in a suitable environment.

Given this is how high-performers are, it’s possible to plot a similar graph for low performers as a diagnostic report. If such a diagnosis is right, it can provide a valuable strategy for managers and employees to work towards improving employee performance. For example, if an employee lacks skill and motivation but fits very well within the work environment, a manager can focus on skills training and suggesting therapies to increase motivation. Similarly, if an employee is unable to acquire the skills for a particular job, focusing on motivation or changing the environment for that employee may not fix the low performance issue.

Based on the past 14 years of my time in the industry, I’ve shared some symptoms of low performance that I’ve seen. You can use this as an exercise sheet to plot your diagnosis of the low performer’s chart.

  1. New hire too slow to ramp up: You were impressed by a candidate during the interview. The same interview process was used to find many high performers in the past. But this employee is struggling to ramp up.

  2. Knowledge vs. application: You have an employee who’s always studying job related skills and shows a great deal of interest in training, but can’t translate the knowledge into application.

  3. Complains about the lack of time: You have an employee struggling to produce the best outcome. In your one:one meeting, the employee always complains about lack of time even after you give him/her the time.

  4. Frequently runs into conflicts: Your dream employee is causing too many conflicts within the team.

  5. Can’t write thoughts down: You have an employee who’s unhappy about things. When you want them to write it down, they are unable to produce a crisp writeup about the problems and potential solutions.

  6. Doesn’t make hard decisions timely: You have a leader that stays away from making hard decisions. In their mind, they want someone else to make them take the hard decision.

  7. Isn’t aware of better ways to solve old problems: You have an employee who’s expert in certain ways of delivering work but lacks awareness of newer and more efficient ways.

  8. Doesn’t take initiative: You encourage and reward employees to take initiatives that help the business. Yet, you see an employee is not taking part.

The good news is, once a diagnosis is accurate for such symptoms of low performance, you can move ahead and implement a concrete improvement strategy for your employee. The clarity of this simple method helped me in many situations.

I hope this helps you, too.

I Just Love to Read Code

This week I took a couple of days to focus on my mid-thirties. It got me thinking about what I actually love doing as a hobby. The list is fairly small, walks with the family, soccer, tennis, coffee with friends, reading, writing, car videos, and Netflix.

Thinking about how I spend my idle time, I realized that I missed two things that I actually love and spend a fair amount of time on: stock markets and reading open-source code.

Ruby on Rails got me into the habit of reading code. Back in the day before bundler, the whole Ruby on Rails framework was in the plugins directory of our code. It had little documentation. A lot of methods accepted options as arguments, and looking at the source code was often the only way to find what options were available. However, that necessity soon turned into a hobby for me, and I keep reading source code of open-source projects just for fun. GitHub mobile UI helps a lot.

To name a few, I read some part of the source code of Ruby on Rails, Ruby, RSpec, BackboneJS, ASP.NET MVC, Golang, UnderscoreJS, VueJS, ReactJS, Terraform, Chef, Jenkins, Lucene, ElasticSearch, Docker, and many other popular tools and libraries.

During this mid-thirty reflection, I decided to start a YouTube channel called ReadCodeWithSohan where I’ll be screencasting as I read some part of an open-source project. Today, I posted my first video on reading the JQuery source code, which I didn’t really read until now. I hope you enjoy it.

Working With Me

Since I commonly work with a diverse group of people, this post is a way for me to broadcast some quick tricks that may help our working relationship. When we work together, let’s do this:

Trust is only earned when it’s mutual. Let’s work with this shared belief.

Context is everything. Most importantly, show me why we need to work on something. If the answer isn’t clear in your mind, we can work together to build a crisp context.

Solutions for problems you’re raising. Ideally, you’ve already thought about a few solutions with some measure of feasibility for each, and you come with an open mind. I know you care about the problem only if you spent some time thinking about a few solutions.

Clarity between your evidence and assumptions. Assumptions are often essential to move ahead, but treating assumptions as evidence can be harmful.

Output is the focus, our egos aren’t. That said, let’s be mutually respectful.

Time is managed. Let’s manage our expectations given the time we have. Blaming the shortage of time shows a lack of skill.

Writing is a great thought exercise and good write-up scales infinitely. Let’s write and rewrite our thoughts, specially when we’re unclear about those.

Standup comedies, biography, soccer, politics, investment, tech, leadership, Netflix, movies, and similar topics are cool. Sorry, not much of a Hockey or Basket Ball fan. Mostly a passive music listener and happy to listen whatever is on radio or a YouTube playlist.

If you have a similar list, I’d be happy to read and respect it. Please send along.

How About Full-Stack Micro-Services?

patched quilt

Source: Audrey on Flickr

I think nobody knows how to stitch together an app with full-stack micro-services. I have the following open-questions if you disagree. Of course, if we could send people to moon, we could solve these problems. But the question is, is it worth and should your team solve these problems? Especially, for small teams?

  1. How to render the UI from tens of independent micro-services into the same web page?
  2. How to ensure the JS and CSS libraries are compatible within all of the independent services?
  3. How to aggregate logs from the services to be able to trace a user / request / transaction?
  4. How to measure and reduce overall latency and spinner-fatigue?
  5. Which off-the-shelf framework can be used for achieving the above?

Headless micro-services are easy to build, but in many ways are similar to integration over database. It helps scaling teams, but even if you have many teams, I’d say extract service where it makes sense instead of adopting that model as the default choice.

I suggest being careful about following conference talks and blogs on how cool micro-services are. It may work for big and gig tech. They don’t always turn a profit! Your small teams are more likely to drown in worthless complexity from a micro-service architecture.

Notes From the VoxxedDays Banff, 2019

VoxxedDaysBanff2019

Gone are the days when I used to go to a ton of local developer meetups. After Polyglot YYC, this was the second conference I joined this year. This was a 50/50 event for me in terms of like vs. dislike. Here goes my notes:

Ix-Chel Ruiz presented the keynote to start the day. Her storytelling style reminded me of WWII movies, where a lot of cloudy and gloomy scene keep getting darker and darker, ending with a deep sense of sadness. Ix-Chel used quotes from the likes of Prof. Galloway and Elon Musk to tell a story about how the modern tech is creeping into our private lives. I wish she had prepared for the keynote a little better. She paced it so slowly yet ran out of the material half-way through the slot. In a way I liked the early finish because I just wanted to feel optimistic about the future.

The stunning view of Banff landscape brightened up the mood a bit:

banff

There were 4 other talks on the same day and I liked the talk on Vue.js and Vuetify by Dave Paquette. Since, I use VueJS at work, it was easier to relate to. Dave showed us how he’s using Vue.js to render a real-time collaborative UI for complex seating requirements at FlyOver Canada.

Among other talks, there was one on data management for machine learning and another one on Kotlin, I liked these talks but didn’t fully relate with the topics.

There was a talk on using Kafka for geo-fencing applications. The presenter could’ve spent more time on the parts of kafka that makes it a compelling case for using in geo-fencing applications. Instead, he delved into the code, java, and jar files.

There was another talk on using EventStore for event sourcing. The presenters set the story saying that they had an opportunity to try new technologies while rewriting a “monolithic” application and they used EventStore with “multi-services” to implement it so it’s easier to evolve the system. I’d love to see some evidence of the desired outcome, that’d be motivating. But in the end it was about the “how” instead of the “why”.

One thing to note is, even for talks that could be more engaging, I still learned quite a few new things.

Beyond the talks, I like going to conferences as it opens the opportunity to meet new people and some known faces. In hindsight, had I known about this event before the talks were finalized, we’d have submitted a few proposals from Cisco. Looking forward to the next round for an opportunity to share some innovative work we’re doing at Cisco.

What We Learned About Feature Flags in Five Years

Looking at our git logs from Cisco AMP for Endpoints Console, I see that we introduced feature flags back in January, 2014. The reason I got interested in it is because even after all these years of use, today I had to build a new concept on our feature flag code. If you’re already using feature flags or thinking about adding feature flags to your project, this experience report may be helpful.

switchboard

Photo credits to Michael Newton

Back in 2014, we were growing as a team, but wanted to keep working on a single shared code. We perceived that the productivity gain of multiple teams working on a shared code would outweigh cross-team dependency issues. As we started working on multiple features in parallel, mostly independent with different release dates, we saw unfinished work on one feature was blocking the release of a completed one. After some research, we decided to introduce feature flags in our code.

First, we read Martin Fowler’s article on this topic as a guideline. Today, we have 195 feature flags in production. Over time, we have extended the use of feature flags with new concepts and I wanted to document it here for everyone. Fowler’s blog also published a more detailed and updated post later. The taxomony used here is different from Fowler’s because I find the following to be more relevant for our product.

  1. Database stored: We store the feature flags in the main database so that the features can be toggled without needing a code deployment.
  2. Cached: Feature flag lookups are cached for performance.
  3. Temporary vs. permanent: We mark some feature flags as temporary when the primary goal is to incrementally release code to production. Temporary feature flags are regularly cleaned once the feature is complete. 13/195 currently used feature flags are marked temporary.
  4. Self-serve: We tag some feature flags as self-serve where users need to opt-in to use the feature.
  5. Limited availability: For self-serve feature flags, we tag some features as limited availability. It allows us to release self-serve features to selected customers.
  6. Globally enabled: We have a mechanism to globally enable or disable a feature flag. 131/195 feature flags are currently marked globally enabled. This number varies by deployed environments.
  7. Enabled for all, but: We have a mechanism for enabling a feature flag for all but some specific targets.
  8. Multi-target: Sometimes we attach a single feature flag to multiple domain objects such as tenant, user, subscription tier, etc.
  9. Hierarchical: We use a fallback mechanism for feature check. For example, the check if a user have file upload permission, we check it for the specific user, then fall back to the tenant it belongs to, and finally fall back to the feature itself being globally enabled.
  10. Code generator: We use a single-command code generator to introduce a new feature flag to our code. It takes care of the database migration, seed entry, and code references.
  11. Circuit-breaker: For integration with external services, we’ve used feature flags as a circuit-breaker to gracefully handle third-party downtime.
  12. Environment-flags. We deploy the product to multiple geographic environments, including a private cloud model. Certain features behave differently based on the deployment. Using feature flags make it easy to develop and test such differences before deploying to each target environment.

There are reusable libraries and services such as LaunchDarkly that provide rich APIs and user interfaces for feature flags. At this point, even with all the aforementioned concepts, our custom implementation of feature flag is quite straight-forward and easy to evolve. It has been a key ingredient for our frequent iterative deployments with 6 teams working on diverse features in parallel on the same product.

Software Architecture Is All About Ugly Boxes and Lines - My Wishlist

In my last post, I claimed software architecture is all talk and no show. When we have a visible one, it’s a bunch of poorly drawn boxes and lines. I don’t have a problem with boxes or lines, but I do like beautiful drawings.

Despite many standards, we still mostly use whiteboard drawing of boxes and lines for sharing software design as we build new systems or introduce new team members. Where it sucks is the lack of evolution and context of the rest of the system that’s not drawn on the board.

A digital repro of software architecture diagrams often happen in PowerPoint or similar tools that allow us to draw boxes and lines. This process is so rough that people just give up.

At work, I have been using WebSequenceDiagram. While it’s still not an eye-candy, I like the fact that you can draw a diagram from using plain text. Consider this as an input to create the accompanying diagram:

1
2
3
4
5
title Toilet Flush System
User -> Flush Lever: Push
Flush Lever -> Outlet Valve: Open
Outlet Valve -> Toilet Bowl: Water
Outlet Valve -> Inlet Valve: Open

Sequence Diagram

While this text to sequence diagram is a great achievement for a tool, I don’t see such tools for software architecture diagrams. Here’s my wishlist of features that I’d want in a software architecture tool:

  1. Text input. Allows us to easily create the diagrams and use all the version control features.
  2. Map like UX: Allows us to easily transition between higher and lower level components.
  3. Beautiful.

Do you know any? Do these requirements make sense?