Introduction

For those not familiar with Ambient TVs, it’s a way to soften the jump from the edge of the TV screen and its immediate surroundings to provide a more immersive experience. I had some LED lights lying around and decided to see if it was possible to control the lights through code and, in turn, make my computer screen an ambient monitor. Whilst I wanted to use it for my monitor, it can be used anywhere and with whatever colours you can send it, including other features your lights may have such as audio reaction or random patterns. I’ve been meaning to write this post for a while as I’ve been using it on an earlier monitor but I never got round to adding it to my new monitor so I documented as I went along for anyone who might find it useful. So let’s get to it! (Please note, LED lights are likely to be Bluetooth Low Energy (BLE) so your computer will need to support BLE in order to interact with them). Full code is on GitHub.

High Level Steps

1. Find out what commands the LED light’s Bluetooth receiver accepts
2. Send commands to the LED lights via my computer’s Bluetooth
3. Obtain the dominant colour of the current screen
4. Send the dominant colour to the LED lights

Prerequisites

• Bluetooth-supported RGB LED lights and accompanying app (I’m using Android, iOS would likely require an alternative approach than the one described here but should be possible using Wireshark directly to monitor Bluetooth traffic). I've attached these lights to the back of my monitor
• Wireshark
• Android’s SDK tools (specifically adb.exe)
• Developer tools (I’ll be using Python 3.10, though any 3.x versions should work, but the principles should be the same whatever for language you prefer)
• A device to send BLE commands from (e.g. a laptop that supports BLE)

Getting Bluetooth data

The first step we need to do is ensure that the app that comes with the lights is working as expected. This can easily be tested by running the light’s original app and making sure that the lights react accordingly depending on the on/off/lighting buttons you’re pressing on your app. We do this because we will shortly be pressing and detecting the specific codes sent to the Bluetooth receiver on the lights.

There are two approaches that I could take. One was to decompile the app’s JAR file and find the codes that were being sent, but I wanted to learn more about the Bluetooth protocol so opted to log all Bluetooth activity on my Android and extract it from there. Here’s how:

1) Enable Developer Options on your Android device

2) Enable Bluetooth HCI snoop log (HCI stands for Host-Controller Interface). You can find this option in Settings > System > Developer or search for it in settings as in the image below

3) We now need to perform specific actions so we can identify what each action sends to the light’s Bluetooth receiver. I’m going to keep it simple to On/Red/Green/Blue/Off, in that order, but if your lights support other features, you can toy around with those too.

4) Run the app and press On, Red, Green, Blue, and Off. It may also be useful to keep an eye on the approximate time to make it easier to filter if you have a lot of Bluetooth activity on your device.

5) Turn Bluetooth off so we don’t get any more noise. In the following steps, we’ll analyse the Bluetooth commands and, as we know the order of what we pressed, we can find out which values correspond to which button press.

6) We now need to access to the Bluetooth logs on the phone. There are several ways to do this, but I will generate and export a bug report. To do this, enable USB Debugging in the phone’s Settings, connect the phone to the computer, and use the adb.exe command line tool.

               adb bugreport led_bluetooth_report

7) This will generate a zip file on your computer’s local directory with the filename “led_bluetooth_report.zip”. You can specify a path if you prefer (e.g. C:\MyPath\led_bluetooth_report”)

8) Within this zip are the logs that we need. This may vary device to device (please comment if you found it elsewhere on your device). On my Google Pixel phone, it was in FS\data\misc\bluetooth\logs\btsnoop_hci.log

9) Now we have the log files, let’s analyse them! To do this, I decided to use Wireshark so start Wireshark and go to File...Open... and select the btsnoop_hci log file.

Whilst it may look daunting, let’s make it easy for ourselves to find what we’re looking for by filtering the BTL2CAP on 0x0004 which is the Attribute Protocol in the Wireshark source code. The attribute protocol defines the way two BLE devices talk to each other, so this is what we need to help find how the app talks to the lights. You can filter the logs in Wireshark by typing btl2cap.cid == 0x0004 in the “Apply a display filter” bar near the top and press Enter

Now we have filtered the log, it should make looking for the commands easier. We can look at the timestamps (Go to View…Time Display Format…Time of Day to convert the time if it’s the wrong format). We want to look at the Sent Write Command logs as those are the ones where we sent a value to the lights. Assuming your most recent time is at the bottom, scroll down to the last five events. These should be On, Red, Green, Blue, and Off in that order with Off being last.

Take note of the Destination BD_ADDR as we'll need that shortly and don on your Sherlock Holmes hat as this is where we need to unlock the pattern of how the colours and on/off commands are encoded within the message. This will vary depending on the light manufacturer but here’s the list of values I got for my device:

• On: 7e0404f00001ff00ef
• Red: 7e070503ff000010ef
• Green: 7e07050300ff0010ef
• Blue: 7e0705030000ff10ef
• Off: 7e0404000000ff00ef

These are clearly hexadecimal values and if you look carefully, you’ll see there are some fixed patterns. Let’s split the patterns out as this should make things much clearer.

• On: 7e0404 f00001 ff00ef
• Red: 7e070503 ff0000 10ef
• Green: 7e070503 00ff00 10ef
• Blue: 7e070503 0000ff 10ef
• Off: 7e0404 000000 ff00ef

For those familiar with hexadecimal values of pure red, green, and blue, you’ll know that the values are #FF000, #00FF00, and #0000FF respectively, which is exactly what we can see above. This means we now know the format to change the colours to whatever we want! (or at least to what the lights themselves are capable of). We can also see that On and Off have a different format from the colours and similar to each other with On having f00001 and Off having 00000.

That’s it! We now have enough information to start coding and interacting with the lights.

Connecting to LED lights

There are three key things we need:

• The address of the device (this is the Destination BD_ADDR from above)
• The values to send to the device (the hexadecimal values obtained above)
• The characteristic we want to change. A Bluetooth LE characteristic is a data structure that essentially defines data that can be sent between a host and client Bluetooth devices. We need to find the characteristic (a 16-bit or 128-bit UUID) that refers to the lights. There are some commonly used assigned numbers that can be found here but unless the device conforms to those, they could be using a custom UUID. As my lights aren’t in the assigned numbers list, let’s find it via code.

I’m using Python 3.10 and Bleak 0.20.1. Ensure Bluetooth on your computer is turned on (no need to pair with the device, we’ll connect to it through code).

# Function to create a BleakClient and connect it to the address of the light's Bluetooth reciever
async def init_client(address: str) -> BleakClient:
    client =  BleakClient(address)  
    print("Connecting")
    await client.connect()
    print(f"Connected to {address}")
    return client

# Function we can call to make sure we disconnect properly otherwise there could be caching and other issues if you disconnect and reconnect quickly
async def disconnect_client(client: Optional[BleakClient] = None) -> None:
    if client is not None :
        print("Disconnecting")
        if characteristic_uuid is not None:
            print(f"charUUID: {characteristic_uuid}")
            await toggle_off(client, characteristic_uuid)
        await client.disconnect()
        print("Client Disconnected")
    print("Exited")

# Get the characteristic UUID of the lights. You don't need to run this every time
async def get_characteristics(client: BleakClient) -> None:
    # Get all the services the device (lights in this case) 
    services = await client.get_services() 
    # Iterate the services. Each service will have characteristics
    for service in services: 
        # Iterate and subsequently print the characteristic UUID
        for characteristic in service.characteristics: 
            print(f"Characteristic: {characteristic.uuid}") 
    print("Please test these characteristics to identify the correct one")
    await disconnect_client(client)

I’ve commented to the code so should be self-explanatory but essentially, we connect to the lights and find all the characteristics it exposes. My output was:

Characteristic: 00002a00-0000-1000-8000-00805f9b34fb
Characteristic: 00002a01-0000-1000-8000-00805f9b34fb
Characteristic: 0000fff3-0000-1000-8000-00805f9b34fb
Characteristic: 0000fff4-0000-1000-8000-00805f9b34fb

A quick Google of the first two UUIDs shows this refers to the name and appearance of the service which is irrelevant for us. However, the third and fourth seem the most suitable with the third (0000fff3-0000-1000-8000-00805f9b34fb) being the write characteristic according to this page.
Excellent, we now have the characteristic we need for this particular device to write to with a value (the colour hexadecimal).

Controlling LED lights

We finally have all the pieces this we need. At this stage, you can get creative with what colour input you’d like to use. You could, for example, connect the lights to a trading market API to change colours according to how your portfolio is doing. In this case, we want to make our monitors ambient aware, so we need to obtain the dominant colour of the screen and send that through.

There are many ways to do this so feel free to experiment with whatever algorithms you would like. One of the simplest approaches would be to iterate every X number of pixels across the screen and take an average while more complicated solutions would look for colours human eyes perceive to be more dominant. Feel free to comment any findings you’d like to share!

For the sake of this blog post, I’m going to keep it simple by using the fast_colorthief library’s get_dominant_color method.

'''
Instead of taking the whole screensize into account, I'm going to take a 640x480 resolution from the middle. 
This should make it faster but you can toy around depending on what works for you. You may, for example, want
to take the outer edge colours instead so it the ambience blends to the outer edges and not the main screen colour 
'''
screen_width, screen_height = ImageGrab.grab().size #get the overall resolution size 
region_width = 640
region_height = 480
region_left = (screen_width - region_width) // 2
region_top = (screen_height - region_height) // 2
screen_region = (region_left, region_top, region_left + region_width, region_top + region_height)

# Create an BytesIO object to reuse
screenshot_memory = io.BytesIO(b"")

# Method to get the dominant colour on screen. You can change this method to return whatever colour you like
def get_dominant_colour() -> str:
    # Take a screenshot of the region specified earlier
    screenshot = ImageGrab.grab(screen_region)
    '''
    The fast_colorthief library doesn't work directly with PIL images but we can use an in memory buffer (BytesIO) to store the picture
    This saves us writing then reading from the disk which is costly
    '''

    # Save screenshot region to in-memory bytes buffer (instead of to disk)
    # Seeking and truncating fo performance rather than using "with" and creating/closing BytesIO object
    screenshot_memory.seek(0)
    screenshot_memory.truncate(0)
    screenshot.save(screenshot_memory, "PNG") 
    # Get the dominant colour
    dominant_color = fast_colorthief.get_dominant_color(screenshot_memory, quality=1) 
    # Return the colour in the form of hex (without the # prefix as our Bluetooth device doesn't use it)
    return '{:02x}{:02x}{:02x}'.format(*dominant_color)

The code is commented so hopefully it should be clear as to what’s happening but we’re taking a smaller region of the screen from the middle then getting the dominant colour from that region. The reason I’m taking a smaller region is for performance; fewer pixels would need to be analysed.

We’re almost there! We now know what to send it and where to send it. Let’s finish the last major part of this challenge which is to actually send it. Fortunately, with the Bleak library, this is quite straightforward.

async def send_colour_to_device(client: BleakClient, uuid: str, value: str) -> None:
    #write to the characteristic we found, in the format that was obtained from the Bluetooth logs
    await client.write_gatt_char(uuid, bytes.fromhex(f"7e070503{value}10ef"))

async def toggle_on(client: BleakClient, uuid: str) -> None:
    await client.write_gatt_char(uuid, bytes.fromhex(ON_HEX))
    print("Turned on")

async def toggle_off(client: BleakClient, uuid: str) -> None:
    await client.write_gatt_char(uuid, bytes.fromhex(OFF_HEX))
    print("Turned off")

As we discovered from the logs, each colour has a fixed template so we can use f-strings to hardcode the common part and simply pass a hexadecimal of a colour for the value in the middle. This can be called from our loop. On and Off had unique hexademicals so I created individual functions and passed in a constant value that contained the relevant hex.

 while True: 
         # send the dominant colour to the device
         await send_colour_to_device(client, characteristic_uuid, get_dominant_colour())
         # allow a small amount of time before update
         time.sleep(0.1)

And there we have it, our Bluetooth LED lights are now controlled by the colours on the screen creating our own Ambient Monitor.

You can see the full code on GitHub which has a small amount of infrastructure code that wasn't specific to this post. I’ve tried to comment the code to be self-explanatory but feel free to ask any questions or make suggestions.

Hopefully this gives you an idea on how you can start getting creative with your LED lights.

Future Improvements

• Multi-colour - My light strips can only have one colour at a time, but I have another set that can have four quadrants, each with their own colours. This means it could be possible to have sections of the monitor match four sections on screen giving an even more accurate ambience setting. Those lights run on Wifi instead of Bluetooth and could be a future project.
• Brightness – to keep it simple, I just looked for the colour changing and the on and off commands. However, this can easily be improved by detecting the brightness control commands and throwing that into the colour algorithm.
• Performance - As we want to get the lights to change in realtime, performance is critical. There are some complex algorithms to detect which colour would be considered the most dominant, especially when perceived by humans (which leads to a whole world of colour conversions). However, since this needs to run quite quickly, there needs to be a balance between performance and accuracy. A future improvement could be to try and access the graphics card directly to read from the buffer rather than analysing the pixels on the screen directly. If this is possible, you would also eliminate the time taken from the graphics buffer to the screen which could optimise the reaction of the lights.

Feel free to comment below if you have any feedback or questions.