Content tagged with "Softeng"

Inundated by 'quick questions' that prevent you from carrying out your main role? Slack has made it very easy to invade your focus time and before slack people would turn up at your desk in the office and say "can I borrow you for a sec...?"

The big problem, especially for software engineers and deep knowledge workers, is that these "quick questions" and just being borrowed comes with context switching cost. The person who asked it may not realise that their 2 minute question just cost you half an hour of work time because you need to get back into the zone (or a nerdier analogy - you need to page the contents of your brain out to disk and loading it back into memory is slow work...)

This week has been jam packed with traveling, meetings, events and all sorts! For an introvert like me, it's been pretty hard going pretending to be extroverted and interacting with lots of folks.

The biggest news this week was that my company won another award. A few weeks ago in September we won the CogX award for Best Fintech Company 2023. On Thursday I attended an awards ceremony with my colleague in order to accept the award for Best Emerging Tech Company (on the south coast) 2023.

Introduction

Airbyte is a tool that allows you to periodically extract data from one database and then load and transform it into another. It provides a performant way to clone data between databases and gives us the flexibility to dictate what gets shared at field level (for example we can copy the users table but we can omit name, address, phone number etc). There are a bunch of use cases where this kind of thing might be useful. For example, say you have a data science team who want to generate reports on how many sales your e-shop made this month and train predictive models for next month’s revenue. You wouldn’t want to give your data team direct access to your e-shop database because:

I ran into an interesting typescript/js problem yesterday at work.

The following code snippet was generating an error and a stack trace that was never being propagated up to the caller:

return new Promise(async (resolve, reject) => {
  const data = await this.client.getObject(bucket, path);

  const buf: any[] = [];

  data.on("data", (data) => {
    buf.push(data);
  });

  data.on("error", (err) => {
    reject(err);
  });

  data.on("end", () => {
    resolve(JSON.parse(Buffer.concat(buf).toString()));
  });
});
Read more...