Hack Day - Chat (Khat) GPT

Why did the AI buying assistant start a stand-up comedy career? Because it knows how to find the best "deals" of laughter!

Here is hoping ChatGPT is better at being a buying assistant than it is at writing jokes!

ChatGPT is the hottest topic around the industry and, inspired by a recent tech talk by our CTO, we used our most recent hack day to try and answer some questions about what Kogan.com could use ChatGPT for. Can we train ChatGPT with knowledge of our products so that it can be a buying assistant to our customers? Can we train it with our FAQs and customer support information so that it can help answer customer queries more efficiently? Through a few different experiments, we tried to answer these questions.

So what is ChatGPT?

ChatGPT is an advanced artificial intelligence language model developed by OpenAI that specialises in understanding and generating human-like text. It works by analysing large amounts of text data and predicting the most likely responses or content based on the context provided, leading to highly accurate and coherent interactions.

Where to begin?

The ChatGPT team on our hack day was the largest and involved a good mixture of different types of developers. We split into two main segments - our backend developers focused on training the model with our data and creating an API that could be interacted with. Our mobile developers focused on adding a chat feature to our current app, which would interact with the created API.

Implementation

Training the model

Starting out we investigated which language model would be the best fit for our application. Well-known models like GPT 3.5/4 are best suited for precise answers whereas models like Davinci are better at extended, concise responses. As this was a support-type bot with a general audience, we went with Davinci for a better user experience.

The process of training the bot involved creating a GPT vector index containing all our data using the OpenAI language model we’ve selected. We used the Python lib, LlamaIndex to facilitate this process. We were then able to query our GPT vector index with our questions.

We trained one of the models using product descriptions and product manuals and the second included customer support FAQs and a collection of fake customer support tickets to see if we could train positive interactions into the model.

So, with our models in hand, we jumped in with some questions!!

Immediately we found that general information from our customer interactions with the support team, such as customer name, was being used by the model to assume the user’s name etc. Questions like,

“How can I get a refund on my order”,

Would often come back with responses like,

“Sure , to receive a refund from Kogan…”

The security/privacy implications of a model being trained with conversations were becoming obvious and seemed like a bigger problem for another day, so we had to park the customer support branch of this project for the day.

We focused on the product assistant for the rest of the day, which turned out to be a great decision as we got some great results.

API Setup

In addition to building the model, we also needed to host it somewhere to make it available for the mobile app to consume. Initially, we started building out a lambda function behind an API gateway in AWS to host the application, but in the final hour of our hack day realised that the model wouldn’t fit into the lambda function’s available space, so we quickly pivoted by spinning up an EC2 instance using AWS Lightsail and setting up Flask as our API. It was great to see the team mitigate issues as they popped up like this.

Solution

At the end of the day, the team was able to showcase a working prototype, with our mobile app integrating with the API and returning some interesting responses. The most promising response we got was a question related to safety information contained within one of our product guides.

Q: “Can I use the Estelle Hair dryer in the bathroom?”

A: “No, you should not use the Estelle Hair Dryer in the bathroom. According to the safety warnings, it should not be used in the bath or shower, near a bathtub, washbasin or other products that contain water, and should be unplugged after use and the speed and heat settings should be set as required before use if it is being used in the bathroom.”

Another example of a general question was:

Q: “What options are available in the Kogan washing machine?”

A: ” The Kogan 7kg Top Load Washer offers a range of options, including 7kg clothes capacity, 6 wash programs and memory function, 8 water levels and 9 types of water power, LED display and delay start function, fuzzy logic control and detergent dissolve tech, child safety lock feature, and a drain pump filter to avoid blockages. It is made from high-quality, clear ABS material, making it easy to maintain and durable for everyday use. It also features personal preference settings, allowing you to customise your washing experience.”

At times, the model wouldn’t align the correct information with a given product but this was always good for a laugh.

Q: “Which Kogan toaster would best suit my family of 6”

A: “The Kogan 4 Slice Toaster (Silver) would best suit a family of 6. It features two independent browning controls, seven adjustable browning settings, built-in Defrost and Reheat functions, an ultra-sleek silver design, a weight capacity of 50kgs, and a frame material of wood. It also has a removable crumb tray at the bottom for easy cleaning.”

To improve this we could have provided our data in a more structured format, but for this experiment all our data was unformatted.

One of the main problems we discovered was the time it took to get a response from the API. Responding often took 5-10 seconds to come through and would come through as complete responses rather than each word like the ChatGPT client in your browser does. This was just a limitation of the library implementation and setting, so could be mitigated with a more bespoke solution.

As an added bonus, our mobile developers also took the time to work with SwiftUI, with one even using ChatGPT to assist with this along the way, and we were all impressed at what it was responding with.

Final Thoughts

Overall it was definitely a successful hack day for the ChatGPT team and highlighted the immense potential of the use cases of ChatGPT at Kogan.com. The hack days are a great opportunity to collaborate with other squads within the engineering team which we wouldn’t usually interact with on a daily basis.

Widgeting around Kogan.com Hack Day!

There’s never any rivalry between Mobile developers vs Web developers, right? Well, the intrigue of what happens in the (more exciting) mobile development space piqued the interest of our frontend and backend mates, so a few joined the Kogan Apps team along with our talented UX designer to tackle a project part of a recent Hack Day.

An idea was pitched to develop a Kogan widget for both iOS and Android devices. On any given day, we have multiple deals going on (see below for a screenshot from our website) but we narrowed it down to showcasing our popular ‘Kogan Daily Deal’ - complete with its own countdown timer to firstly, generate excitement and the sense that time is running out and secondly (and importantly in the world of eComm) to induce FOMO to prompt our users to purchase!

Divide and conquer! (How we started)

Consistent with Agile methodology, we created a backlog of tasks and divided them amongst our teammates. The web developers tackled the API, the mobile developers started their widgeting and UI/UX designed away. 

We did manage to squeeze in a quick 101 for the web devs about the wonderful worlds of SwiftUI and JetPack Compose. WD1 (web developer one) tried to relate with how Angular, Vue or React works for the frontend, which is understandable and WD2 (web developer two) amused themselves with the discussion. It did come up later in conversation that WD2 had been a Mobile developer in the past and worked with Objective-C. Despite showing WD1 & 2 the cooler things that can be achieved with SwiftUI and Jetpack Compose, they remained unconverted - but hey, that’s their loss! 

Henceforth, we will relay our learnings and achievements as conquerings!

Conquering #1

For a Mobile developer (MD) that creates the UI and writes code in either Swift/Objective-C or Kotlin/Java, they understand the pains of working with an Imperative Programming Language. However a WD or a frontend developer is blissfully unaware of these issues. They work with FlexBox and CSS, whereas a Mobile or Desktop developer has to work with making the UI as pixel perfect as the designer conceptualized. With a Declarative Programming paradigm like SwiftUI or Jetpack Compose, it makes it easier for our MDs to create the UI, similar to the ease of our teammates using FlexBox and CSS.

When the designs were completed, the next task was to compose the UI using Declarative. As we were waiting for WD1&2 to finish the API that would retrieve the Daily Deal data, static images were used as a placeholder. 

Conquering #2

There was no API in the Kogan-verse to consume the Daily Deal data, so a handy bit of reverse engineering helped WD1 locate the data source such that they were able to create a wrapper to provide the necessary API endpoint that would return Daily Deal structure.

Matching the UI designs accurately on the Widget using SwiftUI was difficult! Where SwiftUI allows for composing new components quickly, it also has its fair share of quirks because the layout gave us some grief! We did end up getting the images to load, but getting the timer to tick away per the three boxed component design proved too challenging to hack on the one day.

Conquered

We completed the widget family for various sizes per the screenshots below. SwiftUI allows for previewing multiple views simultaneously. 

Conquering #3

With SwiftUI or any framework, if we embed data into a scrollView, we have an instant list of multiple items that can be scrolled and viewed. We spent time trying to make scrollView work, only to realize that it crashes the code and SwiftUI does not provide a scrollView to a widget. So the List of Hot Deals had to be limited from a scrollable list to a list of 3 items.

Conquering #4

Making a lot of formatting functions and styles only highlighted that the designs could be improved. While we were working with a declarative way to create the components, having sizes that were fixed ranging from 11 to 22 points was not the best idea. In hindsight, we could have followed the Apple methodology to declare the styles, just like in CSS, and then apply a Heading, Body, type tag etc to an element rather than adhere to strict font sizes. 

Conquering #5

In the designs there were places where the Price did not have decimals and places where the decimal place digits had a smaller font. This got complicated since the textbox that hosted “$479.75” needed to be split into the three components, “$”, “479” and “.75”, then they needed to be formatted accordingly.  So similar to the TimerView above, and to make it customisable, we put a flag in to conditionally display the decimal points to work for both “$479.75” and “$599” type designs. Passing the font sizes and getting the decimal part of numbers is quite painful when dealing with computers storing numbers in 4 bytes. 0.99 would become 0.9900000000000000001 or 1.00 would become 1 and not have any trailing zeros. Our Javascript friends could use strings to take the characters around the decimal point. However we used Swift NumberFormatter to provide a minimum and maximum of 2 decimal places to ensure consistency, and used the suffix command to get the last 2 digits.

Finish

Our conquering process to develop a widget was an interesting one. Experimenting with the quirks of the language and technology enabled us to highlight learnings that were valuable and applicable to our day to day development work as Mobile engineers. And working alongside our WD friends harmoniously to achieve our goals, while showcasing to them the wonderful world of app development, is always a bonus. Onwards and upwards as we look forward to conquering the next hack day!

DECEMBER 2021 HACKDAY (PART 3) v2

Tick tick, boom gate opens!

And by boom gate I am referring to the Kogan.com office car park boom gate. Kogan.com office parking spots are a scarce resource, but the staff members that are lucky enough to have an allocated spot aren’t always using them. Most days there is enough space for other staff members to utilise the parking spots, but there aren’t many remotes to open the car park boom gate. Is there perhaps a way to remotely open the boom gate by using the Kogan app only, without needing an individual remote?

During the recent hack day, team “Kogan Boom” took on solving this problem. We also managed to surprise and perhaps thoroughly confuse some passers-by when we were testing our solution and randomly opening the boom gate from the comfort of the office. This project had everything; electronics, hardware, software, app integration. It was a fun action-packed day for the team and everyone got to contribute.

The Boom Gate project comprised of three software systems:

1. The Boom Box: An ESP32 wired into the physical RF remote control for the Boom Gate.  These remotes are in short supply and why we wanted to build the project in the first place!

2. IoT Cloud: We used AWS IoT Core to register the ESP32 as a ‘Thing’.  AWS IoT made it simple to generate the MQTT authentication certificates that were copied to the Arduino code running on the ESP32.  We created pub and sub topics via AWS MQTT for which to issue commands to the ESP32 controlling the gate and for the ESP32 to communicate back.

3. Web App: A mobile app and web service that enabled authorized users to toggle the car park Boom Gate. Instead of registering this service like another ‘Thing’ - we used AWS boto to publish the message to the MQTT topic and generated a service account for authentication with permissions to publish.

 Overall, when a logged in user hits the ‘Open’ button in the APP, a POST request is made to the web service which then pushes a message to the MQTT topic.  The ESP32 subscribes to the same topic, receives the message and toggles the PIN which connects the circuit on the RF Remote Controller and opens the gate.  The RF signal from the remote conveniently reaches the Boom Gate from within the office!

The Electronics

 The electronics part of the project included connecting a physical car park remote to a wifi enabled Arduino chip, requiring some delicate soldering work. Well it just so happens that Hanna used to do soldering professionally! She had a summer job at an electronics factory a long time ago and got to do a lot of soldering. Hack day impromptu workstation pictured below. It is good to note that neither staff nor equipment was harmed during this process. 

The ESP 32 Arduino chip was programmed using the Arduino IDE. We connected the ESP32 chip using USB Micro cable to our laptop. Testing whether the code is working or not is one of the most challenging parts. Using the IDE, the code is then uploaded to the chip. Then using a multimeter, we checked whether we received the expected HIGH/LOW signals on the right port.

Using the pin diagram we picked pin 5 to write high/low. Thee ESP32 chip was then soldered to the remote which requires a HIGH single to open up the boom gate. Since ESP 32 is WIFI enabled, therefore the code running inside the chip subscribes to relevant AWS Iot pub/sub channels and listens for the relevant incoming messages from AWS ref. https://iot-esp32.workshop.aws/en/module3/configure_esp32.html. Based on the message from AWS, which in our case was plain “open”, a HIGH signal is sent to the remote which then opens the Boom gate. We didn't need to program anything for closing the boom gate as this was handled by the boom gate itself.

The Mobile App and Web Service

Mobile app section of our team worked on the design and implementation of a new “Kogan Boom” app feature, which is only displayed for Kogan.com staff. There is a button to open the boom gate. Since the boom gate closes automatically, a specific button to close it was not needed.

When a user taps the ‘OPEN’ button, a web request is sent to the web server which then publishes a message to the AWS IoT MQTT topic that the ESP32 is listening on.  We achieved this without needing to register the web service as another IoT ‘Thing’ and instead used the AWS Python boto library to publish to the topic via a service account.  This was fairly trivial once we figured out the command to run!

The service account we created was given the AWS IAM access policy: arn:aws:iam::aws:policy/AWSIoTDataAccess so that it could publish to the IoT MQTT queue.

Putting it all together

 The finished hardware prototype turned out to be a rather sketchy-looking box full of electronics and wires. Some of the time the box was even ticking! Perhaps the name “Kogan Boom” was not ideal after all? 

Our hack day project came together nicely in the end and we were able to open the boom gate using the button in the Kogan.com mobile app. All in all, it was a successful hack day full of collaboration, new learnings, laughter and some very tasty donuts! We are very much looking forward to the next one.