Using DeepCode.AI in your CI/CD pipeline

I started using DeepCode.AI a few months ago, initially just sporadically for GitHub projects to get a feeling of the product and eventually leveraging their Visual Studio Code extension for real-time feedback in my IDE.

Now, I just went one step further: Integrating their code analysis into our CI/CD pipeline. DeepCode offers a CLI with a fairly simple command:

So basically all you need to do is adding that command, for example in your CI/CD pipeline.

Configuring GitLab Runners

Since we use custom Gitlab Runners on GitLab.com for our build pipeline, I decided for a custom stage in my .gitlab-ci.yml, making it easy to convert this into a GitLab CI template or simply making it a bit more re-usable.

First, ensure you have a build image with python 3.6+ installed. Install the deepcode python package. Get the DeepCode API Key from their UI, and plug that into your pipeline, for example via GitLab environment variables.

Adding AWS KMS

In case you run your GitLab runner on AWS, you could use AWS KMS to encrypt credentials such as the DeepCode API Key:

The .gitlab-ci.yml file can now be enhanced with the decryption step of the encrypted DeepCode API Key. This eliminates the need of using environment variables or other customization in your GitLab project.

This is the .gitlab-ci.yml build step I ended up using:

Doing this was already worth the effort. In one of our projects it found a potential UnhandledPromiseException from a promise chain, which was easy to fix. But such bugs will now be stopped before code gets merged.

Operations Teams vs. Engineering Teams

Or: How to use incentives to stop constant escalations

There are many differences between a team focused on operations and a team with an engineering approach. I could go on for hours (or likely, for days) with a rant. But let’s narrow it down to a few key aspects.

Effective vs. Efficient

Operations teams are highly efficient. They get a support ticket, they jump on it, they fix it, the declare success. They have KPIs on how quickly they solve tickets, how many tickets they solve. They have leader boards on identifying the hero of the week. Management supports this further by promoting the fastest. This is an incentive to beat these metrics. Operation teams actually script and automate every overhead to become more efficient. In short, they automate all the waste that shouldn’t even exist in the first place. This is true in all types of organizations, for example an operations-led data team or a customer support team.

Engineering teams on the other hand hate support tickets. They address the root cause. Was it a bug? Let’s fix it. Was it unclear documentation? Let’s clarify it. Was it a hard-to-use user interface? Let’s simplify it. They think in eliminating waste instead of automating it. They are not efficient, but highly effective in what they are doing.

Low vs. high noise level

Operations teams love escalations. Love when something is on fire. This is excitement. There’s something to jump into and start solving. Once it’s solved, this is often recognized in Emails reaching a large audience on the heroic efforts. And the worst? A week later they have another escalation for the very same reason!

Engineering teams are led by people who avoid escalations at all costs. And if they happen, they look into the root cause, address the root cause to solve them for the long-term. They will still thank their team for heroic efforts on a weekend, but they’ll recognize twice as much the work that was done to prevent the same thing to happen again.

Better vs. more

is a bit of a continuation of my rant. But it’s still a key aspect of a team’s culture. Engineering teams want to have better tools, better-skilled team members, better tests. Operation teams want more. They want more people on their team to handle even more tickets. More escalations to have more processes. In many companies, more often means more budget and more power on the decision table. One team puts the solution at the center, another one the team itself. It’s a very human reaction to want more, and for leaders who are human to incentivize more. Giving recognition for less is hard. But today is a good day for you to start doing this.

What’s next?

To round it up a bit, terms like “DevSecOps” got invented to build development, security, and operations into the team’s responsibility. But it’s actually much more. Product ownership, requirements engineering, project management. We actually need a new name that captures the broad aspects fo a team’s work, but I leave this for another day.

Stop calling them direct reports

Or acting like a coach is a choice.

The word direct report sounds hierarchical. That she has to deliver a report I can sign off. That I’m in charge of her. That I am the authority of her actions. This might to some degree be useful in a process-oriented manufacturing shop floor (and even there it starts to be outdated). But it’s far from what should happen in an office full of creative work, where people are measured by the creative value-driven output and not by the number of hours they show up.

Creativity needs diversity in thoughts and approaches. Reports deliver the opposite.

Creativity is where humans shine compared to robots. Authority from a manager over a direct report treats them like robots.

Creativity needs flexibility of where, when and how the work is being done. Reports ask for standardization, like passing a test.

Stop being a career manager who treats the team like a crowd of direct reports. Choose to be a career coach so they can grow their very own personalized progression. Matching their needs and aspirations.

Who will you coach to success today?

Reading list 2019

After summarizing what I read in 2017 and 2018 respectively, it’s time to eventually reflect on my past 12 month, Jan – Dec 2019. Also this year, I learned a lot.

I started the year with All Marketers are Liars by Seth Godin. It’s about telling stories, and telling them in a true and inspiring way. Coming directly from the master story teller, it contains a lot of ideas – while all of these will need practice (and likely a lot of failures on the way).

Then, an excellent read with Principles: Life and Work by Ray Dalio. It came before Ray put much of his content so broadly online and on social media. And even today, I’d recommend a thoughtful read of this content instead of just grasping bits and pieces from tweets. Ray adresses decision making at its root, for your personal life and at the workplace. Deciding once and following your rule book reduces your daily decisions as well as increases the likelyness for success.

After this, I came across The 5 Elements of Effective Thinking by Edward B. Burger and Michael Starbird. The analogy to earth, fire, air, water and the quintessential element matches chapters understand deeply, fail to succeed, be your own socrates, look back look forward and transform yourself. After having read Kahneman’s Thinking Fast and Slow a few years ago, this is another perspective on how to learn thinking and apply it more effectively and how this can have a highly positive effect on your performance.

Culture Infusion: 9 Principles for Creating and Maintaining a Thriving Organizational Culture by Kerry Alison Wekelo is a very practical approach on how culture affects a work environment. She is telling a story from her influence at a company called Actualize where she helped transform the company to adopt a focus on people, work-life balance and aligning the award system with goals. Nothing out of band what one would expect, at the same time she highlights well how challenging the implementation of these principles into a corporate culture can be.

Then I derailed to Dare to Lead by Brene Brown. Actually, it’s an excellent book, but the concept of vulnerability and how she describes many examples, didn’t resonate with me. I dragged on this book for months. That said, the concept of vulnerability is highly valuable, and I took away many little insights. If you want to get into this topic, listen to her TED.com talk The Power of Vulnerability.

Then came The Trillion Dollar Coach by Eric Schmidt, Jonathan Rosenberg and Alan Eagle. I was truly inspired by the approach that Bill Campbell took, first as sports coach, then as coach for many big players in the Bay Area, including Google. A person so determined, so clearly and straightforward minded and focusing on building relationships over almost anything else. This book isn’t really teaching you to coach better, and is more inspirational and actionable – but I turned the pages quickly.

A fun read was Ultralerning by Scott Young. He took the challenge to complete the MIT in less than 2 years and avoid speaking English for one year to learn four new languages. He outlines a practical guide to learn through focus, drill and retention. It contains fun stories of other ultralearners, shows how we could do much more while learning and truly shows that the sky is the limit with regards to acquiring knowledge and use it in practice.

Eventually I couldn’t escape the many recommendations to read a Yuval Noah Harari book. I chose 21 lessons for the 21st century. The expectations were probably too high to be overwhelmed. However, I truly like his fact-based approach to think about challenging topics like war, god, liberty or meaning. He conveyed the information very objectively, and some points had a lasting impact on me. However, the topic range is enormous and while he did his research and came to a pragmatic conclusion for each of them, I’m sure there are many other possible inputs to consider and sharpen each of the topics further.

As a long-time reader of Farnam Street’s blog, I had to grab a copy of The Great Mental Models: General Thinking Concepts. I just love their focus around mental models and our challenges when making decision. This book didn’t reveal anything truly new since most of the content has been made available online, but it was a different experience to use dedicated reading time to go through them a bit more structured than a randomly timed tweet. For me, time worthwhile spent.

After having read Finite and Infinite Games by James Carse, I had to compare this to a showmaster’s version of it. The Infinite Game by Simon Sinek. I had low expectations, except a very practical approach to the infinite game thinking. With that in mind, I was glad that Simon Sinek mentioned James Carse version in the first few pages to get this off the plate. Great. And then, yes, it was followed by many, many examples from the business world. Some of them interesting, some of them an interpretation of Simon Sinek into this model. So overall, a relativley fast read and a lightweight introduction compared to James Carse’s harder to read and abstract version.

As the last book of the year, I got Start Finishing by Charlie Gilkey referred. As many time management books, also this was filled with good advice. One of them stuck with me: Thinking of different time frames – yearly plans, monthly plans, weekly plans and well, daily plans. I mean, everyone knows this and everyone applies this already. Right? True. Not a novelty. But very nicely framed, and I started applying this to my personal work schedule with initial tangible output. So that alone buys the book for me.

Podcasts and Newsletters

I continued following the daily blog post from Seth Godin. After having followed him for a few years, nothing outstanding. But every time I am impressed how concise he’s able to make a point and I feel like I need this daily push.

I listened to most podcasts from Farnam Street, impressed by their ongoing focus on decisions and mental models. The guests have impressive background and interesting stories.

I started reading through the long blogs from the Silicon Valley Product Group, providing useful context around OKR setting, product management and team alignment.

So what’s coming in 2020?

Well – more or less a continuation of above. So far I can reveal that I finished Black Box Thinking by Matthew Syed and started Leadership is Language by L. David Marquet.

What are yours? (Tell me on Twitter.)

A boring place

Or how to spark creativity.

Recently, I wrote about A calm place. A place without hectic, without weekend work, in a well-aligned team. Obviously not everything is perfect, but still: What’s next?

Continuation of above. Have no team changes, no big changes around your goals and mission, and simply show up and execute well. Do this for over half a year. Repeat and repeat. Become boring.

And eventually, a first voice raises. What’s next? How can I strive? How can I advance? What will the team be doing in a year from now? Questions you hoped to hear a long time ago are eventually asked.

If it’s boring, kids start playing with the simplest things available. The same for your team members. Boring places allow for thinking time. And then, individuals start to question the status quo. They start to think about their next moves. Creativity is born.

A calm and boring place allows for creativity, for experimentation, for failure. It’s now time to explore this territory again. Back then, it was a reactive failure mode. This time, it’s a proactive, adventurous exploratory mode.

Be surprised what curiosity might bring to you and your team.

AWS ElasticSearch anonymous access from a VPC

It’s a bit tricky to find how to configure anonymous access to AWS ElasticSearch from a VPC. Especially when you start from this AWS support article and dive deep into configuring IP-based access and policy conditions.

Even if you look at the AWS Console, there are pre-configured access policies. None of them saying “Enable anonymous access” or related hints.

And at the same time it’s too obvious.

On the UI, it’s saying Do not require signing request with IAM credential.

That’s it. Click this. Apply without modification. There are no IP restrictions required or even possible from within the VPC for anonymous access. (Which is different for public AWS ElasticSearch domains.)

And if you’re already using CloudFormation or other forms of scripts, here’s the short version as access policy:

A calm place

Or the importance of operational excellence

When I joined my current team last year, there was a lot of excitement going on. First, the schedule was determined by people outside of the team. Almost everything being built was based on ad-hoc, seemingly urgent support requests. The team was busy rushing features and bug fixes out to get through the backlog, which only kept increasing. Second, the on-call rotation was being passed through the whole team of almost 20 engineers. The weekly rotation hit you roughly twice a year. It meant sleepness nights, waking up multiple times. But it was over soon, and your motivation to fix things dropped to zero – merely hoping the situation will improve until your next turn.

It was time to challenge the status-quo. Persistently asking why we woke up people just to snooze an alert for a few hours. Why we had little ownership. Why the motivation was low to fix the root cause.

I challenged myself to measure my contribution by my very own, subjectively measured “team calmess index”. It started with giving support. Standing behind bold decision to only alert during office hours for many cases. Removing the large team-wide responsibilities and make them specific to a sub-group. And also finding time to fix some of the underlying system’s errors. Within weeks, my subjective “team calmess index” improved a lot. Without a change in technology, without a change in skills. Only the right management support, focus and ownership.

It has become a calm place. As the numbers tell.

Number of incidents per week with 3 month moving average.

Promotion by PowerPoint

Or how to avoid killing people with your slides

Earlier this year I came across the blog post Death by PowerPoint: The slide that killed seven people. Not only dramatic, but sad reality in companies today.

Here’s a typical encounter: At the end of the meeting, the meeting owner asks the fearful question: “Who wants to summarize?” Silence. Avoiding eye contact. After a few silent seconds that feel like minutes, a shy hand goes up. The new employee. Ah! Still innocent.

Two days later, a link to the meeting notes arrive in the inbox. Keeping it there. Unread. Postponing clicking on it until 5 minutes before the follow up meeting. Time passes. On the way walking to the meeting, swiping through it on the phone. Shocked.

The new employee took initiative. By creating something that wasn’t just a report. Something that wasn’t meeting the existing low standards. It was establishing something new. A new way of thinking. Taking initiative by interpreting the meeting. Making suggestions. Drafting solutions. Stating decisions.

The whole “report” ended up as the baseline of the new project. How it was done. What was done. Who works together. After a few minor tweaks, everyone nodded. Key decisions usually taking hours of discussion were taken away in a few slides. By the new employee!

It’s your chance to change culture and influence your team. Make bold and clear statements in executive summaries, team reports and presentations.

Ethics as technical solution

Or how to avoid another GDPR meeting

I recently blogged about Regulations as bad. And how regulations help big corporate. However, also big corporate is being challenged, and the usual solution is to hire a team of lawyers and have them schedule endless meetings across the company.

Regulations are one thing. They demand the absolute minimum with an abolute maximum of bureaucracy. This is an opportunity to take an engineering approach. To find purpose. To do better than demanded. And also to do it right from a technical standpoint.

So, what can an engineer do?

Start with asking the question. Asking the question of the one person who is not at the meeting table. The customer. Your customer. What would she think if you’d discuss how to find a loophole in the regulation instead of taking care of her desire to treat her data with respect and the deserved respect and privacy?

I’m glad we agree. Let’s dive into simple technical solutions: End-to-end encryption (in transit and at rest). This might have been hard a few years ago (remember self-signed certificates in a local intranet?), but in todays world of cloud-computing there are no excuses of not using HTTPS with a free certificate or enable the option for encryption at rest of storage, backups or databases.

Use an external authentication provider to authenticate access to your systems. Don’t re-invent the wheel. Auth0, Facebook Connect, Azure AD and many more have long solved OAuth and SAML authentication and can be connected to directory services for federated logins. Most 3rd party systems support that, and custom services should leverage those systems and just deal the straight forward call of validating JWTs.

Encryption and authentication alone don’t solve the privacy problem, especially if everyone working in your company has access to the data. Tokenizing data containing personal information is moving it however to a next level, especially for central log or analytics systems. Those system often contain the full contact details of a customer. But why? Why do you want to expose your analytics system to data breaches that can be executed with a simple “select email from customers” query? Why isn’t it enough to just store the postal code and country, and potentially a token to assign a useful primary key without personal information to the customer address?

And finally, there are still regulatory requirements you need to meet, such as having a data protection officer assigned. But hopefully that’s just a small administrative overhead to everything you already do. All the things you do much better than regulation demands.

Disclaimer: Above is a simlification trying to make a point. There’s more you can and should do from a security and privacy standpoint. And there are a few more things you must do for aligning with regulatory authorities.

Regulations are bad

Or how they help big corporate. And how to navigate them as a small company.

I don’t think it comes to a surprise to anyone. But regulations are a hot topic recently. They are created with all the good intent. But in the end, they only help the largest corporations, and sometimes destroy the smallest companies.

So, let’s start with a popular example. GDPR. And there’s one very visible side-effect of it – the cookie consent banner that pops up on most websites these days. Adding this banner is relatively straight forward from a software engineering perspective, but it’s either mission impossible or an expensive exercise for small companies. Small companies (outside the tech world) don’t have much idea what a cookie is, what privacy implication it has, but they still run a website. Often leveraging WordPress or Shopify etc. It was easy to setup those websites self-service. But all of a sudden they have to choose between million of plugins that claim to have the best cookie consent banner for their small website. And once they activated the plugin, they won’t know whether it even complies. Or they contract a software engineer, and all their margins of the past month are gone for a few cookies. With all the good intents, the small can only lose.

Let’s move to food. Labels are popular. From sustainable fishing to organic food to socially responsible sourcing. Labels have good intent. They help to identify a standard quickly. Example fair trade. The fair trade price is meant to pay the world market price plus a premium. A premium that should reach the producer, or better the worker at the farm. The deal is, that the premium must first be used to improve work conditions so workers benefit in a way that can easily be audited. Sadly, the premium isn’t a lot, and often the certification process and keeping authorities happy for renewals is more expensive than the premium alone. So for the smaller ones it could actually mean less money for the workers. And the very small can’t even afford the certification in the first place. The small can only lose.

Big corporate love regulations, and their complexity. They have the lobby on their side so regulations are ending up being highly complicated, the lawyers to find the right loophole, and the money to actually work within those regulations. And they have independent lawyers on their side who try to find the small who don’t (or better can’t) comply, simply to make money from them. The small can only lose.

So, what’s your choice? Regulations cover the minimum only. Loopholes included. Then there’s ethics. It’s what should be done. It’s way above regulations. Write them down for yourself as your principles. Make decisions following these principles. Your customers will understand them when being consistent. And they’ll be loyal as long as you stay true to your principles.