This is the foundation post for the rest of my Home Assistant energy series (if I ever get to it). Before I could automate around electricity, I obviously needed a (real-time) view of what’s actually flowing in and out.

Some context first

• I have solar panels (with a Huawei inverter installed by the previous owner)
• Smappee EV charger (bad purchase, it’s quite dumb and has a terrible UI and Home Assistant can’t control it),
• and I recently got 2x Marstek batteries (great hardware, cheap, and the worst software ever, truly terrible software. Oh and support started ghosting me after I lost my patience around Lunar New Year — they just will not reply to any of my help messages anymore).
• I drive a Polestar 4 (first edition, 2025)

note: the original Huawei inverter was dead when I moved in and likely never worked, don’t think the previous owners ever noticed & it got replaced under warranty after a lot of chasing and nagging Huawei directly. Rensol couldn’t move it forward and I basically asked to be cc’ed in the ticket and got my PA to chase Huawei with all het wrath. Great service from both Huawei and Rensol…

So that’s that… ;)

SlimmeLezer

… And more context.

I use a SlimmeLezer+ with my P1 port (Fluvius smart meter). As ESPHome has significantly changed since I got mine and it wouldn’t compile on a recent release, here’s my working config. ¯\(ツ)/¯

The SlimmeLezer used to be entirely powered by the P1 port but brownouts did happen; my initial config tried to limit memory and WiFi usage in an attempt to prevent these brownouts)… As this is no longer much of an issue (see below), I’ve stopped optimising that. But YMMV depending on your setup.

As the Marstek CT also uses the P1 port and I needed a splitter: I started off with a passive splitter but it was quite clear from the get-go that there were tons of issues (both meters were powered over P1). It’s been a while but I think roughly half the messages didn’t go through to SlimmeLezer (IIRC the message would just be forwarded across the two ports but would be too weak to be consistently read or something). I then got an active P1 splitter. As I only had one power plug available, I’d power either the splitter or the SlimmeLezer over a USB-C PSU… But what really fixed it and brought stability was doubling the power sockets: the splitter is powered over P1 power, both the SlimmeLezer and the Marstek CT have their dedicated power in via USB-C.

Honestly, I had a lot of issues with the Marstek CT, to the point they actually shipped me a replacement unit. In hindsight I don’t think it was broken, more a mix of crap software + P1 power issues + delayed/laggy Marstek dashboards + …

The setup, end to end

                ┌─────────────┐
                │ Solar panels│
                └──────┬──────┘
                       │ DC
                ┌──────┴──────┐
                │   Huawei    │── fusion_solar (cloud)
                │   inverter  │
                └──────┬──────┘
                       │ AC
        ───────────────┴───────────────  ← house AC bus
        │              │                │
  ┌─────┴──────┐ ┌─────┴──────┐  ┌──────┴────────┐
  │ Marstek    │ │ Fluvius P1 │  │ Smappee EV    │
  │ Venus E ×2 │ │ smart meter│  │ Wall + CTs    │
  │ (1× CT003) │ └─────┬──────┘  └──────┬────────┘
  └─────┬──────┘       │ RJ12, DSMR     │ (cloud; read-only)
        │ Modbus       │                │ 
        │ TCP   ┌──────┴──────┐         │
        │       │SlimmeLezer+ │         │
        │       └──────┬──────┘         │
        ▼              ▼                ▼
        ┌────────────────────────────────┐
        │       Home Assistant           │
        └────────────────────────────────┘

SlimmeLezer is fire-and-forget (at least as long as you don’t update ESPHome). Huawei is a cloud-poll. Marstek is local Modbus (but Modbus/battery BMS or firmware does crash and gets stuck. Restarting the integration sometimes fixes it). Smappee, cloud-only, can only read data (as opposed to telling it when to, for example, start/stop charging my EV).

Sensors

What you get out of it, sensor-wise:

sensor.slimmelezer_power_consumed — instantaneous import (kW)
sensor.slimmelezer_power_produced — instantaneous export (kW)
Per-phase power in / out
Per-phase voltage and current
Cumulative energy import + export per tariff
Gas consumption (if you have a gas meter on the same connector)

That’s all I need to know whether I’m paying or selling at any given moment.

The SlimmeLezer YAML

Stock Zuidwijk firmware with three small tweaks and a few changes to get it to compile on the latest ESPHome at the time of writing):

1. output_power: 14dB on the WiFi to fit inside the P1 250mA budget without rebooting (before getting an active splitter and dedicated power).
2. rx_buffer_size: 1200 on UART (down from default 1700). Belgian P1 telegrams are well under 1200 bytes; saves ~500 bytes of heap which the ESP8266 will thank you for.
3. A “Last Restart” and “Last Time Sync” template sensor so I can spot if the device has flatlined.

---
substitutions:
  device_name: slimmelezer
  timezone: "Europe/Brussels"
  domain: ".iot.internal"
  wifi_fast_connect: "false"

esphome:
  name: ${device_name}
  name_add_mac_suffix: false
  project:
    name: zuidwijk.slimmelezer
    version: "2.0"
  on_boot:
    then:
      - if:
          condition:
            lambda: return id(has_key);
          then:
            - lambda: |-
                std::string key(id(stored_decryption_key), 32);
                id(dsmr_instance).set_decryption_key(key.c_str());
          else:
            - logger.log:
                level: INFO
                format: "Not using decryption key. If you need to set a key use Home Assistant service 'ESPHome: ${device_name}_set_dsmr_key'"

esp8266:
  restore_from_flash: true
  board: d1_mini

mdns:
  disabled: false

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  fast_connect: "${wifi_fast_connect}"
  domain: "${domain}"
  # Powersaving for brownout due to 250mA restriction P1
  output_power: 14dB

logger:
  level: INFO
  baud_rate: 0

api:
  reboot_timeout: 15min
  encryption:
    key: !secret encryption_key_slimmelezer
  services:
    service: set_dsmr_key
    variables:
      private_key: string
    then:
      - logger.log:
          format: Setting private key %s. Set to empty string to disable
          args: [std::string(private_key).c_str()]
      - globals.set:
          id: has_key
          value: !lambda "return private_key.length() == 32;"
      - lambda: |-
          std::string pk_str(private_key);
          if (pk_str.length() == 32)
            pk_str.copy(id(stored_decryption_key), 32);
          id(dsmr_instance).set_decryption_key(pk_str.c_str());

ota:
  platform: esphome
  password: !secret ota_password
  on_error:
    then:
      - logger.log:
          format: "OTA update error %s"
          args: ["x"]

dashboard_import:
  package_import_url: github://zuidwijk/dsmr/slimmelezer.yaml@main
  import_full_config: true

time:
  - platform: homeassistant
    id: homeassistant_time
    timezone: "${timezone}"
    on_time_sync:
      then:
        - text_sensor.template.publish:
            id: last_time_sync
            state: !lambda 'return id(homeassistant_time).now().strftime("%Y-%m-%d %H:%M:%S");'
        - if:
            condition:
              lambda: 'return id(device_last_restart).state == "";'
            then:
              - text_sensor.template.publish:
                  id: device_last_restart
                  state: !lambda 'return id(homeassistant_time).now().strftime("%Y-%m-%d %H:%M:%S");'

globals:
  - id: has_key
    type: bool
    restore_value: yes
    initial_value: "false"
  - id: stored_decryption_key
    type: char[32]
    restore_value: yes

uart:
  baud_rate: 115200
  rx_pin: D7
  # Drop DSMR/UART buffers from 1700 → 1200 (Belgian P1 is fine at 1200; saves ~500–600 bytes).
  rx_buffer_size: 1200

dsmr:
  id: dsmr_instance
  max_telegram_length: 1200

sensor:
  - platform: dsmr
    energy_delivered_lux:
      name: "Energy Consumed Luxembourg"
    energy_delivered_tariff1:
      name: "Energy Consumed Tariff 1"
    energy_delivered_tariff2:
      name: "Energy Consumed Tariff 2"
    energy_returned_lux:
      name: "Energy Produced Luxembourg"
    energy_returned_tariff1:
      name: "Energy Produced Tariff 1"
    energy_returned_tariff2:
      name: "Energy Produced Tariff 2"
    power_delivered:
      name: "Power Consumed"
      accuracy_decimals: 3
    power_returned:
      name: "Power Produced"
      accuracy_decimals: 3
    electricity_failures:
      name: "Electricity Failures"
      icon: mdi:alert
    electricity_long_failures:
      name: "Long Electricity Failures"
      icon: mdi:alert
    voltage_l1:
      name: "Voltage Phase 1"
    voltage_l2:
      name: "Voltage Phase 2"
    voltage_l3:
      name: "Voltage Phase 3"
    current_l1:
      name: "Current Phase 1"
    current_l2:
      name: "Current Phase 2"
    current_l3:
      name: "Current Phase 3"
    power_delivered_l1:
      name: "Power Consumed Phase 1"
      accuracy_decimals: 3
    power_delivered_l2:
      name: "Power Consumed Phase 2"
      accuracy_decimals: 3
    power_delivered_l3:
      name: "Power Consumed Phase 3"
      accuracy_decimals: 3
    power_returned_l1:
      name: "Power Produced Phase 1"
      accuracy_decimals: 3
    power_returned_l2:
      name: "Power Produced Phase 2"
      accuracy_decimals: 3
    power_returned_l3:
      name: "Power Produced Phase 3"
      accuracy_decimals: 3
    gas_delivered:
      name: "Gas Consumed"
    gas_delivered_be:
      name: "Gas Consumed Belgium"
  - platform: uptime
    name: "SlimmeLezer Uptime"
  - platform: wifi_signal
    name: "SlimmeLezer Wi-Fi Signal"
    update_interval: 60s
  - platform: template
    name: "Free Heap"
    lambda: |-
      return (float)system_get_free_heap_size();
    unit_of_measurement: "bytes"
    accuracy_decimals: 0
    entity_category: diagnostic
    icon: "mdi:memory"

text_sensor:
  - platform: dsmr
    identification:
      name: "DSMR Identification"
    p1_version:
      name: "DSMR Version"
    p1_version_be:
      name: "DSMR Version Belgium"
  - platform: wifi_info
    ip_address:
      name: "SlimmeLezer IP Address"
    ssid:
      name: "SlimmeLezer Wi-Fi SSID"
    bssid:
      name: "SlimmeLezer Wi-Fi BSSID"
  - platform: version
    name: "ESPHome Version"
    hide_timestamp: true
  - platform: template
    name: 'Last Restart'
    id: device_last_restart
    icon: mdi:timeline-clock
    entity_category: diagnostic
  - platform: template
    name: "Last Time Sync"
    id: last_time_sync
    icon: mdi:clock
    entity_category: diagnostic

Drop that into ESPHome, set your WiFi/encryption secrets, flash, and the device will appear in HA’s ESPHome integration with all the sensors above.

I’m very happy with the SlimmeLezer+ and got my parents one too.

I have quite a few sites where some Unifi U6+ Access Points fail to upgrade with a generic update failed message.

Marquis WiFi Vergaderruimte 4 update failed.

I've tried everything, from ssh'ing, factory resetting with set-default, to manually upgrading with upgrade, etc.

Nothing worked. I thought I had a bunch of bad APs (and many had their warranty expire by a few weeks or months).

As a last resort, I decide to reach out to Ubiquiti's support. First line wasn't very helpful, but as the case was escalated, I've managed to recover and properly update the APs.

The trick is to ssh into the device and run the following code:

curl -fs https://dl.ui.com/firmwares/uap/jw/U6Rescue.sh | sh

The device will reboot and pop by up in the dashboard after 2-3 minutes. Then try to update again, and all should work.

Not really sure what's going on here but ...

I recently got a WP6003 Air Box (Aliexpress). The official product page seems to be offline but can be accessed through Archive.org.

It's quite straightforward to integrate in a Bluetooth Proxy and Home Assistant thanks to Omar's work.

The major problem I have with it, is that the Bluetooth (4.0) range is terrible. Neither my Mac Mini nor my Bluetooth Proxy could connect with it if I moved it from the office to the kitchen (had it on top of the fridge, so some metal there) that's roughly 10ish meters away.

Sensor wise, I've only had it working for a few days, and so far, it doesn't seem to be very accurate. The temperature is 1-2° below my other sensors... But hey, it's quite cheap and the previous tool I linked has a way to calibrate it (but I haven't tried that yet).

In the screenshot, Airquality level converts bytes into meaningful sensor readings:

• Temperature: Extracted from bytes 6 and 7, then divided by 10 to get the temperature in degrees
• TVOC (Total Volatile Organic Compounds): Extracted from bytes 10 and 11, then divided by 1000 to get the concentration in parts per
• HCHO (Formaldehyde): Extracted from bytes 12 and 13, similarly processed to TVOC.
• CO2 (Carbon Dioxide): Calculated from bytes 16 and 17, adjusted by subtracting 150 to correct the baseline.

Each of these values is then published to different sensor IDs within ESPHome, allowing Home Assistant to display and use this data.

The function returns the size of the data packet as a float, which can be used for debugging or validation. In my case (and assume yours as well) it's 18. I've marked it as internal: true in my config, as it's not useful to show once it's working.

My config for ESPhome can be found here.

You can use this tool to read out the sensor (and find the MAC address) from Chrome.

Guess it's a bit hard to show (and for some reason Firefox refused to take a screenshot of the entire page) but this is my current view in Home Assistant.

Note that I have since added 3 more ATC Xiaomi thermometers and hidden Airquality levels.

In my Home Assistant quest and simplifying (not sure if that's actually happening) and getting more control (aka less reliant on the cloud and 3rd parties) I've been playing with Zigbee.

Zigbee is quite cool because it's a mesh network. As I used it before with my Philips Hue bulbs, I was eager to expand and play with Zigbee.

First thing first. Not all routers are equal. I've got these two smart power plugs as an example (Grey and Blue colour, v2 with HomeKit support) and they act weirdly. They don't seem to route sensors properly, and randomly turn off whatever is plugged in.

The one that seems to work reliably well is this orange Zigbee plug.

Compared to my bulbs (more on that here) the sensors do detect them the two aforementioned power plugs, and do connect, but generally little to no data flows. I'm not entirely sure what the issue is (bad antenna, bad routing software) but the consensus on the internet seems to be: add more routers, don't trust some routers (Ikea and Hue seem decent), and not all endpoints are very smart and pick the best router (so may need to repair or force it to join a specific router). Unlike WiFi, in this case it seems that more is better.

When setting up the network, it's also recommended to first set up and pair the routers and then the endpoints (battery-powered sensors).

And the network takes a while to stabilise (and all routes to be learned). Some people even suggest waiting one or two days for the dust to settle.

Moving routers screws up a lot, too. I've had sensors stop working (and refusing to pair again) after moving some routers and turning some off -- routers that weren't used by said sensors. So really odd stuff.

As always, with Aliexpress, it's very much unclear what the brand/model is, but according to Z2M, all three are TS011F, but two of the three are definitely different (has Apple Homekit support and the plugs are bigger).

The energy (kWh) and Power (W) reporting works well, though, on all three models.

Can see when a charger is plugged in, and charging a MacBook Air.

I'm buying EU plugs because I'll be relocating back to Europe in the next 6 months. So I'm slowly getting rid of my UK smart plugs and replacing them with EU plugs.

Note that these plugs are not approved for (official) sale in Belgium though: child safety is not up to spec and the grounding is not approved (i.e. needs to be a pin). 

In my Home Assistant quest and simplifying (not sure if that's actually happening) and getting more control (aka less reliant on the cloud and 3rd parties) I've been playing with Zigbee.

Zigbee is quite cool because it's a mesh network. As I used it before with my Philips Hue bulbs, I was eager to expand and play with Zigbee.

I have several bulbs, including: Hue Iris, Hue Go, some random bulb, a reading spot, and Hue Play (3x).

My Home Assistant setup uses the Skyconnect. I've updated the firmware (on a Mac) to use Ember firmware as ezsp will be deprecated soon(™).

At the time of writing you can't update the firmware directly from HA just yet, you need another device.

As I had two Zigbee networks (one for Hue, and one for my sensors and Ikea bulb) it made sense to merge them (and have one less connected bridge): it would reduce interference, and the bulbs all act as a router, so that would greatly improve my network coverage and stability.

Generally speaking, it went quite smoothly: delete the devices in the Hue app (it factory resets them and puts them in pairing mode) and then pair them in Zigbee2Mqtt (I use that over ZHA).

However, my main issue was that my Zigbee bulbs were unreliable: it sometimes worked, but often it lagged or timed out. Especially when controlling multiple bulbs at once (such as a big room such as the living room) or sending multiple commands in sequence (on/off/on/off).

In hindsight, my sensors were unreliable as well: there was data missing, and it didn't properly broadcast/update every 5 minutes. I first thought it was just because they were cheap Chinese (TuYa) sensors using battery-power. If the temp or humidity didn't change enough, it wouldn't broadcast.

I often saw errors such as these in Z2M:

2024-05-26 22:29:02Publish 'set' 'state' to 'Hue Play Right' failed: 'Error: ZCL command 0x0017880104d89bc4/11 genOnOff.on({}, {"timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed ({"target":29608,"apsFrame":{"profileId":260,"clusterId":6,"sourceEndpoint":1,"destinationEndpoint":11,"options":4416,"groupId":0,"sequence":110},"zclSequence":244,"commandIdentifier":11} timed out after 10000ms)'

But the annoying thing (in debugging, and thinking the network "needs to settle") was that it often worked just fine, and then randomly started breaking down/time outing when I thought it was finally solved.

I almost gave up and went back to a Hue bridge, when I asked for some advice on Discord. The reason was the channel...

First off: check the Zigbee channel of your Hue. It's likely 25. Z2M uses channel 11 by default.

There are 4 channels that are getting the least WiFi interference. These are called ZLL channels (whatever that stands for): 11, 15, 20, 25. However, it seems that 11 is quite terrible, and I would not recommend using it.

My Z2M sat on Channel 11 (which is Channel 1 for WiFi). I live in Singapore, in an apartment building, with a shit-ton of WiFi, hence the massive interference and lag on Zigbee.

Also, 2.4Ghz WiFi is generally 20Mhz channel width, whereas Zigbee is 2Mhz, so one WiFi channel overlaps many Zigbee channels. 

You can modify the channel (via the config file in /homeassistant/zigbee2mqtt/configuration.yaml) by adding the following and restarting the Z2M container:

advanced:
  [...] # there should already be a ton of things
  channel: 25

The problem with changing the channel, is that it requires repairing (some of) the Zigbee devices. And it's a total mess to repair Hue bulbs once they are no longer on a Hue bridge.

There's no need to remove the devices from Z2M as that will wipe their configs (I believe). It's possible that removing the devices from Z2M will put them in pairing mode again (for Hue bulbs, see below) but I couldn't get a confirmation.

Ikea has the 6x power off/on sequence that resets them, but Hue dropped support for something similar in some firmware upgrade (why?!?!).

TouchLink is another option, but that didn't work for me at all.

Some Hue bulbs can be reset by long pressing (30-60seconds) the power button (for example the Hue Go), but what worked best for me is pairing them again on the Hue bridge using the serial (force pair), deleting them again, and pairing on Z2M.

However, I didn't have the serial of two bulbs: one Hue Play has the label missing, and the Hue Iris didn't have any label at all. I thought I would be forced to go and purchase a Hue dimmer/remote control (that comes with TouchLink and allows resetting the bulbs)...

However, suddenly, after an hour or two of messing around, the Hue bulbs that weren't repaired just started working again.

So it seems that if they are disconnected from their bridge for a while, they start changing channels until they find the bridge again. I guess there's a chance, with enough patience, that I didn't have to repair my Hue bulbs (not sure about Ikea, and I would definitely not trust my Chinese Power plugs and sensors to come back online).

The sensors are quite easily repaired (long press the pairing button or pinhole and done -- don't forget to set Z2M in Permit join)

Other stuff you can try if the range or network sucks can be found here.

However, if it kinda works, you better not mess with it... :)

I've finally started working with Home Assistant and while the learning curve has been steep, it's been fun.

I'm going to post some random notes (mostly for myself) on things I discovered, and I need to keep in mind.

For example, I have set up an ESP32-c3 Bluetooth Proxy for a Mi ble thermometer.

I've assigned the ESP device to an area: IT cabinet. However, the BLE sensors are scattered across the house (in different areas). Took me a while to figure it out, but you can assign entities to specific areas.

Head to Entities and select the entity config, and at the bottom, unselect use device area, and select the correct area.

Tadaaa, you can now assign the sensors to the correct area, and they'll show up in the right dashboards/cards.

The proxy config can be found here.

I have set up an ESP32-c3 Bluetooth Proxy for a Mi ble thermometer (and I have a few other BLE sensors that I haven't managed to set up yet).

The proxy config can be found here.

Be sure to modify the variables to match whatever you want to do. I'm using static IPs because auto discovery is unreliable (hostname.iot.internal).

You'll also need to add the !secret variables into the secrets.yaml file.

Note that there's an issue with Bluetooth and WiFi on single cores (i.e.: the ESP locks up as it can't do both at boot). The workaround is in that config file.

Use on single-core chips

On dual-core devices the WiFi component runs on core 1, while this component runs on core 0. When using this component on single core chips such as the ESP32-C3 both WiFi and ble_tracker must run on the same core, and this has been known to cause issues when connecting to WiFi. A work-around for this is to enable the tracker only while the native API is connected. The following config will achieve this:

esp32_ble_tracker:
  scan_parameters:
    continuous: false

api:
  encryption:
    key: !secret encryption_key
  on_client_connected:
    - esp32_ble_tracker.start_scan:
       continuous: true
  on_client_disconnected:
    - esp32_ble_tracker.stop_scan:

Esphome

To flash the Mi thermometer, use this flasher and the firmware from here.

Example output of the log (from Esphome in Home Assistant directly, but can be seen from the web interface as well):

You can then add the proxy (and the entities) in Home Assistant. It should update every couple of minutes (the Mi device updates every 5 or so minutes, the Proxy scans every channel every 3-ish minutes).

In the example above, I (at the time of writing) only have one Bluetooth sensor, but it should be quite trivial to add more. The hardest part is finding the right MAC address of the sensors... :)

Seeing this makes me want to get one and keep it for Ila to play on when she's a bit older. I guess this is why every couple of months I go back to check Playdate and am tempted to get one (knowing very well I won't really play it much)...

Raspberry Pi recently launched Bookworm, and with that, a new kernel package.

However, due to the massive changes from Bullseye to Bookworm, they are not suggesting a dist-upgrade.

In the past, we have suggested procedures for updating an existing image to the new version, but always with the caveat that we do not recommend it, and you do this at your own risk.

This time, because the changes to the underlying architecture are so significant, we are not suggesting any procedure for upgrading a Bullseye image to Bookworm; any attempt to do this will almost certainly end up with a non-booting desktop and data loss.

Bookworm — the new version of Raspberry Pi OS

But screw that, ain't nobody got time to reinstall all their RPis in the wild (and in all fairness, I've had zero issues updating; just some messing around with kernels -- hence the post)...

As this is not very well documented, and Raspberry Pi, across all their versions, has 4 different architectures, it's confusing.

There's a quick dirty guide on how to update from raspberrypi-kernel and raspberrypi-bootloader to linux-image-rpi.

However, it's unclear in terms of which kernel to pick, so here goes:

• RPi3 (including RPi4) and newer in 64 bit¹ (aarch64):
  apt install -y linux-image-rpi-v8 linux-headers-rpi-v8
• RPi4 and 400 in 32 bit²:
  apt install -y linux-image-rpi-v7l linux-headers-rpi-v7l
• RPi2 (only 32bit) or 3, 3+, Zero 2 W, Compute Modules 3 and 3+ in 32 bit mode³ (armhf):
  apt install -y linux-image-rpi-v7 linux-headers-rpi-v7
• RPi1, Zero, Zero W, Compute Module 1 (armel):
  apt install -y linux-image-rpi-v6 linux-headers-rpi-v6

This mostly comes from here.

So, in short, this is what worked for me. As I use Wireguard as well, I needed to remove the package first to prevent conflicts.

apt remove -y wireguard
apt autoremove -y

dpkg --purge --force-depends raspberrypi-kernel raspberrypi-bootloader
umount /boot
fsck -y /boot
mkdir /boot/firmware
sed -i.bak -e "s#boot#boot/firmware#" /etc/fstab
systemctl daemon-reload
mount /boot/firmware
apt install raspi-firmware -y

# rpi3+ 64bit
#apt install linux-image-rpi-v8 linux-headers-rpi-v8
# rpi4+ 32bit
#apt install linux-image-rpi-v7l linux-headers-rpi-v7l
# rpi2 and rest
#apt install linux-image-rpi-v7 linux-headers-rpi-v7
# rpi1
#apt install linux-image-rpi-v6 linux-headers-rpi-v6

sed -i.bak '$ a\auto_initramfs=1' /boot/firmware/config.txt 

apt install wireguard -y
systemctl enable wg-quick@wg0.service
systemctl start wg-quick@wg0.service

# reboot

When running apt install <kernel> -- do write down the package it'll install. The listed packages are just meta packages and will pull a bunch of other packages that actually hold the important files. Running apt --reinstall on those meta packages, for example, won't actually reinstall the kernel.

To roll back, you can purge those packages and reinstall raspberrypi-kernel raspberrypi-bootloader (be aware of the path changes in /boot). If you made a mistake, just apt install --reinstall -y <all the packages you wrote down> and it should reinstall everything.

You can see how this script makes that couple very happy.

Quick and dirty script that shows your Mac battery information (health, cycles, etc). If an Apple keyboard or mouse is connected, it'll also display the battery % of those.

# Battery information
battery() {

	if !ioreg > /dev/null 2>&1; then
		echo "ioreg not found. Exiting."
		return 1
	fi

	_ioreg=`ioreg -l`
	_profile=`system_profiler SPPowerDataType`

	MOUSE=`echo $_ioreg -l | grep -A 10 "Mouse" | grep '"BatteryPercent" =' | sed 's/[^0-9]*//g'`
	TRACKPAD=`echo $_ioreg -l | grep -A 10 "Track" | grep '"BatteryPercent" =' | sed 's/[^0-9]*//g'`
	KEYBOARD=`echo $_ioreg -l | grep -A 10 "Keyboard" | grep '"BatteryPercent" =' | sed 's/[^0-9]*//g'`
	CYCLE=`echo $_profile | grep "Cycle Count" | awk '{print $3}'`

	if [ -n "$MOUSE" ]; then
		echo "Mouse: "$MOUSE"%"
	fi

	if [ -n "$TRACKPAD" ]; then
		echo "Trackpad: "$TRACKPAD"%"
	fi

	if [ -n "$KEYBOARD" ]; then
		echo "Keyboard: "$KEYBOARD"%"
	fi

	if [ -n "$CYCLE" ] && [ "$CYCLE" -ne 0 ]; then
		echo "Mac battery "`echo $_profile | grep "State of Charge" | awk '{print $5}'`"%"
		echo "Charging: "`echo $_profile | grep "Charging" | head -n 1 | awk '{print $2}'`
		echo "Cycles: "$CYCLE
		echo "Condition: "`echo $_profile | grep "Condition" | awk '{print $2}'`
		echo "Health: "`echo $_profile | grep "Maximum Capacity" | awk '{print $3}'`
	fi
}

Outputs something similar to this (no mouse or keyboard connected):

nazgul ~ $ battery
Mac battery 54%
Charging: No
Cycles: 224
Condition: Normal
Health: 89%

This works on zsh and may not work in bash.

Content Update

The provided scripts have been updated on 16 Jul 2023. Specifically the SmartStrip part was not working as intended.

I've been a big fan of Betteruptime. I've started using it to monitor all my assets online (websites, DNS, ping, successful script runs) as well as my servers (using heartbeats).

I have a few Raspberry Pi's, and once in a while they hang (not sure why, maybe USB-to-SSD issues or something). Nothing too critical, but annoying.

I've plugged them all on TP-Link Kasa smart plugs to remotely restart them if I had to (once or twice a year).

Note, to confuse everyone, TP-Link also launched Tapo, which... competes with Kasa and is not compatible, but does the exact same thing... ¯\_(ツ)_/¯

After some Googling (actually Kagi'ing) I found out, there's a Python library that lets you control your smart plugs.

So, the idea was born to:

• Check Betteruptime heartbeats, if down, power cycle the smart plug
• Do this at most once per day (in case something else is causing issues)
• Betteruptime heartbeats manage when a device is marked as offline (i.e.: it expects a heartbeat every 5 minutes, but will only consider the device down if no heartbeats are received for 2 hours).
• The bulk of the code had to be written by ChatGPT. Let ChatGPT choose the language (it ended up being a mix of Bash and Python)
• Everything needs to run in Docker (using a cron, the Docker container doesn't daemonise)
• These run on Raspberry Pi's (but of course the RPi can't check itself: so RPi1 checks for RPi2, and vice versa. As these RPis are on different networks (my parent's home, my own home, etc) I had to enable "--net=host" in Docker run to get the correct routes from the host system, but you may not actually need this
• To top if off, sent an email (using Mailgun EU servers) to warn me something broke and it rebooted

So, after some fiddling (half an evening or so) the proof-of-concept worked.

I should probably throw this in a Git repo but shrug. I don't want to give the impression that I'll maintain this and provide support.

Dockerfile:

FROM python:alpine
RUN apk add bash curl jq
RUN pip3 install python-kasa
COPY heartbeat.sh kasa-api.py /
VOLUME /tmp/kasa/
CMD ["/heartbeat.sh"]

Python script kasa-api.py (this works with both smart strips and smart plugs):

import sys
import asyncio
from kasa import SmartPlug, SmartStrip

async def main():
	if len(sys.argv) != 4:
		print("Usage: python kasa-api.py type IP-address outlet-index")
		return

	device_type = sys.argv[1]
	ip_address = sys.argv[2]
	outlet_index = int(sys.argv[3])

	if device_type == "smartplug":
		await control_smart_plug(ip_address)
	elif device_type == "smartstrip":
		await control_smart_strip(ip_address, outlet_index)
	else:
		print(f"Unsupported device type: {device_type}")

async def control_smart_plug(ip_address):
	plug = SmartPlug(ip_address)

	try:
		await plug.update()

		# Retrieve the current state
		plug_state = plug.is_on

		# Turn off the plug
		await plug.turn_off()

		print(f"Turned off SmartPlug at {ip_address}")
		await asyncio.sleep(5)

		# Turn on the plug if it was previously on
		if plug_state:
			await plug.turn_on()

		print(f"Turned on SmartPlug at {ip_address}")
	except Exception as e:
		print(f"Failed to control SmartPlug at {ip_address}: {e}")

async def control_smart_strip(ip_address, outlet_index):
	strip = SmartStrip(ip_address)

	try:
		await strip.update()

		# Retrieve the current state of the specified child plug
		child_state = strip.children[outlet_index].is_on

		# Turn off the specified child plug
		await strip.children[outlet_index].turn_off()

		print(f"Turned off child plug {outlet_index} in SmartStrip at {ip_address}")
		await asyncio.sleep(5)

		# Turn on the child plug if it was previously on
		await strip.children[outlet_index].turn_on()

		print(f"Turned on child plug {outlet_index} in SmartStrip at {ip_address}")
	except Exception as e:
		print(f"Failed to control SmartStrip at {ip_address}: {e}")

# Run the asyncio event loop
asyncio.run(main())

heartbeat.sh -- with example devices. Be sure to fill in the variables (including hb, that's the heartbeat ID you can get from the Betteruptime URL and the IP or DNS hostname of the smartplug):

#!/bin/bash

API_KEY="BetterUptime API token"
BU="https://uptime.betterstack.com/api/v2/heartbeats/" # no need to change this

MAILGUN_API_KEY="Mailgun API token"
MAILGUN_DOMAIN="mg.you.com" # use your own domain

if [[ "$DEVICE" = tyr ]] || [[ "$1" = tyr ]]; then

	# Tyr
	device="Tyr"
	hb=1111
	bu="https://uptime.betterstack.com/team/1/heartbeats/$hb"
	plug_type="smartplug"
	plug_host="smartplug1.kasa.you.com"

elif [[ "$DEVICE" = mammoth ]] || [[ "$1" = mammoth ]]; then

	# Mammoth
	device="mammoth"
	hb=2222
	bu="https://uptime.betterstack.com/team/1/heartbeats/$hb"
	plug_type="smartstrip"
	plug_host="smartstrip1.kasa.you.com"
	plug_index=0 # plug 2 is rly plug 3 because the index counts from 0 to 2 and not from 1 to 3.

elif [[ "$DEVICE" = liana ]] || [[ "$1" = liana ]]; then

	# Liana
	device="liana"
	hb=3333
	bu="https://uptime.betterstack.com/team/1/heartbeats/$hb"
	plug_type="smartstrip"
	plug_host="smartstrip1.kasa.you.com"
	plug_index=1 # plug 2 is rly plug 3 because the index counts from 0 to 2 and not from 1 to 3.

elif [[ "$DEVICE" = eagle ]] || [[ "$1" = eagle ]]; then

	device="eagle"
	hb=4444
	bu="https://uptime.betterstack.com/team/1/heartbeats/$hb"
	plug_type="smartstrip"
	plug_host="smartstrip1.kasa.yeri.be"
	plug_index=2 # plug 2 is rly plug 3 because the index counts from 0 to 2 and not from 1 to 3.

else
	echo "Unknown device."
	exit 1
fi

url=$BU/$hb

send_alert() {
	MAILGUN_URL="https://api.eu.mailgun.net/v3/$MAILGUN_DOMAIN/messages"
	from="kasa@you.com"
	to="alert@you.com"
	subject="Smartplug power cycled: $device"
	body="rebooted device $device!"$'\n'"Kasa IP: $plug_host."$'\n'"$bu"

	# Send alert email
	curl -s --user "api:$MAILGUN_API_KEY" \
		"$MAILGUN_URL" \
		-F from="$from" \
		-F to="$to" \
		-F subject="$subject" \
		-F text="$body"
}

kasa_cycle() {
	echo "Betteruptime heartbeat ($hb) says the service for $device is down, restarting."
	python /kasa-api.py "$plug_type" "$plug_host" "$plug_index"
	# Update the last execution date in the file
	echo "$current_date" > "$file"
}

kasa_info() {
	kasa --host $plug_host
}

response=$(curl -sL "$url" -H "Authorization: Bearer $API_KEY")
status=$(echo "$response" | jq -r '.data.attributes.status')

if [[ "$status" == "down" ]]; then
	dir="/tmp/.kasa/"
	mkdir -p "$dir"
	file="${dir}${device}.txt"

	# Get current date
	current_date=$(date "+%F")
	# Check if the file exists
	if [ -f "$file" ]; then
		# Get last execution date from the file
		last_execution=$(cat "$file")
		# We only want to run this once every 24hrs. If a reboot doesn't fix it, something more
		# serious is going on and likely needs manual intervention. No point spam rebooting the device.
		if [[ "$current_date" != "$last_execution" ]]; then
			kasa_cycle
			send_alert
		else
			echo "Power cycle already executed today."
		fi
	else
		kasa_cycle
		send_alert
	fi
elif [[ "$status" == "up" ]]; then
	echo "Betteruptime heartbeat says the service ($hb) for $device is up."
else
	# this could happen if the heartbeat is paused.
	echo "Unknown status."
	kasa_info
	exit 1
fi

I run Docker with two scripts, a builder (rebuild.sh) and a file that runs it (start.sh). It should rebuild in case a docker cleanup script ran (and deleted dangling containers).

I run this as root and probably shouldn't, but yeah... That'll be for another lifetime.

Be sure to change the paths (/root/git/kasa-api) in both scripts.

rebuild.sh:

#!/bin/bash
cd /root/git/kasa-api # the path where this project exists

git pull > /dev/null

BASEIMAGE=`cat Dockerfile | grep FROM | awk '{print $2}'`
docker pull $BASEIMAGE
docker build -q -t kasa-api .
rm -f /tmp/.kasa/*.txt

start.sh:

#!/bin/bash

if [ -z "$1" ]; then
	echo "Missing device name."
	exit 1
fi

docker stop kasa-api 2> /dev/null
docker rm kasa-api 2> /dev/null

run_kasa() {
	DEVICE=$1
	docker run --net=host -v /tmp/.kasa:/tmp/.kasa --rm -e DEVICE=$DEVICE --name kasa-api kasa-api
}

if [[ $(docker image ls | grep kasa-api) ]]; then
	run_kasa $1
else
	cd /root/git/kasa-api
	/root/git/kasa-api/rebuild.sh > /dev/null
	run_kasa $1
fi

And that's pretty much it. I run this using with cron in /etc/cron.d/. For example (be sure to edit the parameter/device name/path):

#
# cron-jobs for kasa-api
#

MAILTO=root

*/15 * * * *	root	if [ -x /root/git/kasa-api/start.sh ] && [ -f /root/git/kasa-api/start.sh ]; then /root/git/kasa-api/start.sh tyr >/dev/null; fi

I'm sure there must be bugs in this ChatGPT generated code but... so far, it has actually worked.

Just got myself a Flipper Zero because #whynot.

However, the updating process from 0.6x.y to 0.82.3 was not very smooth sailing:

• First it was stuck in synchronising on the iOS app refusing to initiate the update (had to force quit + reboot Flipper Zero to get it to continue)
• After that, once I managed to upload the new firmware, it was stuck on Waiting for SD card after the Flipper rebooted.

The SD card I tried was an old 2Gb Samsung SD I had lying around (have 3-4 of these).

Tried several times, but alas, until I tried a different SD card... Then the update process went fine and the firmware updated as expected.

So looks like this is a sign the SD card is not properly performing (even though format, benchmark and uploading firmware worked fine).

Recently visited Ikea and had to get a Christmas star-light like I used to have during my childhood.

Most of my (non-track) light at home is smart using either a TP-Link Kasa smartplug or Philips Hue (and I want to avoid adding more brands, more apps, and more shit). However, Philips Hue is quite $$$ and not always all that easy to get in Singapore (shady retailers, limited stock, not many options).

I knew that Tradfri was technically compatible with the Hue Bridge, but never made the jump as I just wanted things to work (don't want to mess around with lights). Until today. Figured I may as well give it a go.

Surprisingly, everything worked out of the box. I had to do a factory reset (not sure if that's because I kept the light on for ~10 minutes before trying to set it up, or if this was a returned product as clearly the box had been opened before)...

• Turn on light
• Reset light by turning it off and on 6 times
• Open Philips Hue app and search/add light
• Tada... Should work

Side note: it requires a modern firmware (and I assume all lights being sold today have said firmware, but if for some reason you end up with old stock, you may need to update). To update you need the Ikea Bridge though (or... apparently the dimmers/buttons may work as well -- did not test).

All the controls are working as well (dimming, changing colours, etc).

It's been over two years since my last rant.

I can now happily say this NFC tag hack works. No need to hack your purifier like with their previous kit (I almost purchased the kit -- but eventually decided against it; was it worth the trouble?).

But these NFC tags are significantly easier to use. After ignoring the "your filter is at 0%" for the past ~6 months, it's back at 100% with this. Remember to dust off and clean it once in a while (but even if you don't, the air coming out is still cleaner).

So, entirely, worth it!

Speaking of which -- Ikea has a whole new range of cool purifiers (including built-in a table). Quite cool (and sad that it's come to this with all the pollution)!

Shan uses her iPad a lot, but a lot of the more serious (interior design) work needs to happen on AutoCAD or Photoshop. That is just not going to work on an iPad.

When we're travelling (read: holiday) she's carrying an old Lenovo ThinkPad 13 (great device!) just "in case" she needs to open AutoCAD and edit something minor or read the drawings/dimensions. But honestly, most of the time that device is turned off and dead weight.

But all the above is just an excuse to "I was bored, and I wanted to test something": can I use an old Raspberry Pi (zero W) to remotely wake her Intel NUC, and then use Tailscale to use RD on her iPad? Well, yes I can.

I completed this using:

• Tailscale to remote desktop from anywhere to home
• Cloudflare Tunnels, Access and DNS to have a web interface to wake the desktop
• A Linux device that's always on and in the same LAN, and that'll run a PHP script.

Prep work: enable WOL

First off: enable Wake-on-LAN (WOL) in the BIOS and in your Windows settings. This article explains it for Intel NUCs, but would be similar enough for most devices. The Device Manager pane looked different on our i5 NUC, but was close enough.

On Mac, you just need to enable it in the Energy preference pane, for Linux I have no clue. 🤷‍♂️

Second step: have a working Raspberry Pi (or any Linux device) in the same LAN. This device needs to be turned on 24/7, so use something that uses very little power.

I do have a more powerful RPi4 I wish I could've reused (running Docker and some other "serious" stuff; however it's currently in a different VLAN, and it's quite crucial the Linux device is in the same LAN as the device(s) you want to wake up), so I went with an old Raspberry Pi Zero W that was collecting dust (it used to run pwnagotchi).

On the Linux device, install etherwake. The command to run is quite simply etherwake aa:bb:cc:11:22:33 (= the ethernet MAC address of your device).

If this doesn't wake your desktop, something is wrong and there's no point continuing. Go and troubleshoot.

Install Tailscale and RD

On the (Windows) desktop and your iPad, install Tailscale. Login, and make sure it works by pinging from one to the other.

Then set up Remote Desktop on both (Windows, iPad). You should test and make sure you can properly connect using the LAN IP address and then the Tailscale IP address.

Fun fact: I create a DNS record for all my devices using Cloudflare DNS with the syntax of device-name.ts.yeri.be, so I don't need to ever remember IPs, and can easily ssh or ping devices without having to look up IPs. 

Fun fact side track: I actually have a dynamic script that runs (on Linux) and creates hostname.ts.yeri.be for the Tailscale IP, hostname.wg.yeri.be based on the Wireguard IP, hostname.lan.yeri.be based on the LAN IP. This dyndns script runs every so often and updates IPs if needed. All this is running using Cloudflare DNS and their API. Super convenient.

Nginx, php and etherwake

I'm a 80s kid, so I'll use dirty PHP to run this script. I'm sure I'll go straight to hell for this, but yolo.

Install nginx and PHP (no need for MySQL and other stuff).

etherwake requires root to run (because it needs root access to create a weird magic ethernet packet). Create a file in /etc/sudoers.d/etherwake and add this line:

www-data ALL=(ALL) NOPASSWD: /usr/sbin/etherwake

This will allow www-data (nginx/php) to run /usr/sbin/etherwake using sudo, without password.

In /var/www/html/ create an index.php file with:

<html>
<head><title>Wake on Lan</title></head>
<body>
<p>Wake up <a href="mycooldesktop.php">My Cool Desktop</a>
</body>
</html>

And a mycooldesktop.php (or whatever) file with:

<?php
$output = shell_exec('sudo /usr/sbin/etherwake aa:bb:cc:11:22:33 2>&1');
echo "<pre>$output</pre>";
?>
<p><a href="..">Back</a></p>

It ain't pretty -- but it gets the job done. 

And be sure to edit the MAC address to match your desktop's ethernet MAC address.

Cloudflare tunnels

Install Cloudflare Tunnel (via Zero Trust dashboard).

When creating a new tunnel, the dashboard will give you all needed commands to install the tunnel on your RPi -- but be sure to select the right OS/architecture (arm64? arm? armhf?).

And then point the tunnel to http://localhost. No need to mess around with SSL certs.

Lastly, set up Cloudflare Access (via the same Zero Trust dashboard): create a new application, and make sure only approved users can sign in (i.e.: using a pin code emailed to only approved emails).

You can try it yourself via wol.superuser.one. You won't get in. :)

Optionally, but recommended: lock down Nginx to only allow connections from localhost (127.0.0.0/8 and ::1) if using Cloudflare Tunnels, or Cloudflare IPs if using port forwarding with Cloudflare Access in front. 

To recap

• We used Tailscale to create a VPN network between the desktop and the iPad. The big benefit is that Tailscale works effortlessly across NAT networks without having to open ports,
• We used Cloudflare DNS so we don't need to remember hostnames :),
• We used Cloudflare Tunnels to make sure the RPi web interface is accessible across NAT (without port forwarding) and from anywhere,
• We used Cloudflare Acces and locked down access to the right people using ACLs,
• We used etherwake running on a RPi to wake up devices that are hibernating or turned off.

And that's it really.

PS: technically WoL works with WiFi, but when I enabled WoL on the WiFi adapter, the NUC refused to hibernate/sleep for more than a minute, and kept waking itself up. So, there seems to be some kind of trigger in my network that keeps waking it up. Also, not sure if WoL via WiFi would work if the device is turned off (as opposed to sleep or hibernate). I just ended up using ethernet. 

PPS: both Cloudflare Tunnels and Tailscale use Wireguard tech in the background, so that's really cool. 

This summer, the Lithuanian government went public with an astounding finding. A Xiaomi phone sold in Europe — the Mi 10T 5G — could censor approximately 450 words and phrases, it said. The blocklist wasn’t active, but could be activated remotely. It was filled with political terms, including “Democratic Movement” and “Long live Taiwan’s independence.”

[...]

The accusations, which Xiaomi disputes, clarified just how fraught the West’s relationship is with China’s growing technology power. As China-based tech companies like Xiaomi and TikTok flourish, there’s still no playbook in North America or Europe to deal with their potential to censor or steer culture via algorithms.

“Western countries,” Abukevicius said, “are more and more reliant on technologies, and a big part of those technologies comes from countries which are not friendly, which we don't trust, and it poses risks.”

Source.

"Google developing own CPUs for Chromebook laptops".

Interesting to see that many years after Apple started creating their CPUs for the iPhone (and now laptops/desktops), so many companies are following. Google is not new to building their chips (TPU, Titan (used in security keys and as encryption module for servers/Pixel phones), and likely more), but quite new to more generalised computing CPU for phones and laptops.

And it makes sense -- a lot of the generic CPUs were too generalist and not that great at their job (and are plagued with bugs). It came with heavy power usage. Having a ML/AI chip, a GPU chip, a generalist CPU chip (or two, one focussing on high performance, and one on efficiency, like the M1), one for security/encryption (Titan/T2), etc.

Curious to see how much of a head start Apple really has, and very eager to finally see some real innovation in the CPU space (sorry AMD with Ryzen: too little, too late).

Let's see if Intel and AMD will be able to adapt and reinvent themselves and what it means for ARM (and the ARM IP issue in China), and if other architectures like MIPS are making a chance.

I've noticed my AirPod Max being stuck at ~74% battery and not wanting to charge any further. It's running the latest firmware and I usually charge them on a (legit) usb-A-to-lightning cable connected to my monitor. Even keeping them connected charging overnight would somehow max out at 74%.

There are a few Reddit posts with other people facing the same issue with AirPod Pros (case not wanting to charge further than 74%) but no concrete answer or solution is posted.

For me, what helped, was to use a wall charger (doesn't have to be Apple); it's still using a similar usb-A-to-lightning cable. Letting it charge for a while managed to get it to 100%.

Why 74% and not… 75%? ¯\_(ツ)_/¯

Edit: as some people pointed out, this may actually be Apple's Optimized Battery Charging. I am not seeing any notification though.

I am not sure what is the problem -- I've upgraded Shan's 27" 2k monitor to a 32" 4k monitor and AutoCAD LT recently updated from 2021 to 2022. Shan's been using a 2017 i5 NUC (NUC7i5BNB) with 32Gb RAM using the onboard GPU. Something that should be plenty for a bit of Windows, Chrome, Photoshop and AutoCAD LT.

For the past few weeks, several times a day her screen would black out for half a second or so and then everything should go back to normal. Sometimes it would happen several times in a row, sometimes it wouldn't for hours on end.

I tried several things, including getting a 8k 120hz HDMI cable, using a usb-c to DP cable, changing the GPU vRAM minimum from 128Mb to 2048Mb (quite the hack, as it can't be set in the bios -- requires messing around in the registry. Wow.), updating all the drivers, updating the bios, making sure the NUC was properly ventilated and keeping cool (every full moon the NUC shuts down due to overheating in Singapore if I set the fan to 'balanced' instead of 'cool'), and probably more. And because it happened so sporadically it was quite hard to debug (to the point I sometimes didn't want to believe her as I couldn't see it blacken out).

Shan was getting frustrated (which in turn means I get frustrated)... and I started looking at getting either an eGPU (I can get a free Nividia M6000, just need the enclosure, but the enclosure would be $$), an AMD Ryzen 5 or 7 NUC (no more Intel in this house, but also $$$), or some other refurb (massive) desktop some friend had lying around (HP z240 or something, zero $, but Shan would kill me for having this massive thing on her desk).

Turns out, the quick fix, was to simply disable hardware acceleration.

Run GRAPHICSCONFIG and uncheck hardware acceleration.

This is probably not ideal in the long run (hopefully it's an issue with AutoCAD 2022 that is getting fixed, as opposed to the onboard Intel GPU getting messed up with AutoCAD).

When in doubt… Turn off hardware acceleration.

I rarely use Bluetooth on my RPis. I'm already facing enough issues with my iMac and Mac Mini (it lags, it randomly disconnects in meetings, etc).

My pwnagotchi on the other hand is counting on a BLE network to connect to the internet: for now I am using my iPad, and while that works, it causes my iPad to disconnect from WiFi (because of course, it can only do tethering from a mobile network, not from its WiFi network).

I wanted to explore if I could set up bluetooth tethering/internet sharing from my RPi4 server... But for that BLE had to work! And for some reason BLE was not working on Liana.

[bluetooth]# power on
No default controller available

For some reason no controller was available. The drivers were definitely installed...

apt install bluetooth pi-bluetooth bluez raspberrypi-sys-mods

but hcitool dev ; hciconfig -a weren't returning anything.

After quite some extensive Googling I found the solution...

Check if this returns something:

# ls -l /dev | grep ttyAMA0
lrwxrwxrwx 1 root root           7 Sep  1 15:08 serial1 -> ttyAMA0
crw-rw---- 1 root dialout 204,  64 Sep  1 15:08 ttyAMA0

As opposed to:

# ls -l /dev | grep ttyS0
# (no output)

Then continue to do the following:

# make a backup
cp /boot/cmdline.txt /boot/cmdline.txt.bak
# edit the file
nano -w /boot/cmdline.txt
# edit the first part from
# console=ttySerial0 to console=ttyAMA0
# the line should be something similar to but do NOT blindly copy paste it as you won't be able to boot due to your PARTUUID being different
console=ttyAMA0,115200 console=tty1 root=PARTUUID=6c586e13-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
reboot

After rebooting ... it works!

liana ~ # hcitool dev ; hciconfig -a
Devices:
	hci0	DC:A6:32:B1:0E:79
hci0:	Type: Primary  Bus: UART
	BD Address: DC:A6:32:B1:0E:79  ACL MTU: 1021:8  SCO MTU: 64:1
	UP RUNNING 
	RX bytes:2397 acl:0 sco:0 events:118 errors:0
	TX bytes:2603 acl:0 sco:0 commands:99 errors:0
	Features: 0xbf 0xfe 0xcf 0xfe 0xdb 0xff 0x7b 0x87
	Packet type: DM1 DM3 DM5 DH1 DH3 DH5 HV1 HV2 HV3 
	Link policy: RSWITCH SNIFF 
	Link mode: SLAVE ACCEPT 
	Name: 'liana'
	Class: 0x000000
	Service Classes: Unspecified
	Device Class: Miscellaneous, 
	HCI Version: 5.0 (0x9)  Revision: 0x13b
	LMP Version: 5.0 (0x9)  Subversion: 0x6119
	Manufacturer: Cypress Semiconductor Corporation (305)

liana ~ # bluetoothctl 
Agent registered
[bluetooth]# scan on
Discovery started
[CHG] Controller DC:A6:32:B1:0E:79 Discovering: yes
[NEW] Device 7F:A9:BC:8D:E4:14 7F-A9-BC-8D-E4-14
[NEW] Device 58:EB:19:D8:D4:23 58-EB-19-D8-D4-23
[NEW] Device A4:83:E7:42:79:F6 A4-83-E7-42-79-F6
[NEW] Device 58:7B:24:1B:CC:5C 58-7B-24-1B-CC-5C
[NEW] Device D9:05:9F:DB:55:19 N0163
[NEW] Device 5F:DA:90:34:82:68 5F-DA-90-34-82-68
[NEW] Device 77:2A:1B:11:54:7D 77-2A-1B-11-54-7D
[NEW] Device 42:BF:0B:38:F3:20 42-BF-0B-38-F3-20

Next step is trying to get tethering to work...

Right. With the pandemic and all none of us are going to travel much but still...

About a year ago I purchased myself an OpenWRT router to use on the plane and in hotels.

And so far I really like both the device and the Hong Kong based brand (launching new and updated products, and releasing relatively regular updates for older products). Pick a device that fits your needs (USB powered? LTE? Small form factor?).

The GL-AR750S aka Slate is fully customizable but runs a few nice things out of the box: WireGuard (with a physical button to turn it on or off), OpenVPN, shell access, Tor (requires the latest firmware), IPv6, DoH (Cloudflare only for now), multiple SSIDs (i.e. Guest WiFi), and more.

Oh and I specifically picked this version (compared to other or cheaper ones) because it had both 2.4Ghz and 5Ghz, as well as 3 Gbit ports (1x WAN, 2x LAN).

I use the device on flights, where I connect to the network once in the air, purchase WiFi or use iPass "for one device" and then connect to the interwebs behind my NAT-router from my iPad, phone(s), laptop(s), and even Shan's devices if she is travelling with me.

In hotels, I either connect it to the wired ethernet, if still available (tends to be more stable), or connect it to the guest WiFi and then connect my devices to the router: saves me from connecting to a new network and typing the room number and login/password/family name on every device. And once again hides the true number of connected devices; quite handy trick for those pesky hotels providing free access only to two devices.

Sure it takes a bit of setup every time: find a working USB port, sign in to the web interface, search for new networks if this is a new hotel or I haven't travelled on this airline, connect to said network, sign in with iPass, and optionally enable VPN)...

And once in a while some fiddling with VPN or DNS that's borking up or being blocked by overzealous firewalls.

Also, some in-flight entertainment USB ports don't provide enough power (and/or are often broken -- looking at you Lufthansa in economy) so be sure to carry a couple of these (US-plug works best) -- I've already forgotten one on my last flight from MUC-SIN on LH, but luckily I have pretty easy access to these.

If you travel a lot it's totally worth the money.

All right. With the release of the new RPi4 with 8Gb of RAM I had to get myself one to see if it was already a viable desktop replacement for surfing and emails.

While a SD card works fine for certain tasks (things that don't require a lot of IO) -- for a desktop that's a no-go... It's just too slow.

I still had an old Macbook Pro 13" (2o15?) SSD lying around that was collecting dust. Why not use that one to use as root for the RPi?

This article will focus on making it work on Raspbian first. Technically this should all work on other distros as well, but YMMV seeing all this is still beta.

I use Raspbian Lite: I like to work with minimalstic systems and install just what I need. But technically this should work with any flavour.

But first, let's prep the device.

Case

I already have a RPi4 (4Gb) at home running mostly Docker containers (nginx proxy and a few personal things and Smokeping).

And one of the 'best' purchases I made for the RPi4 was the "Raspberry Pi 4 Model B Aluminium Case" (Lazada, AliExpress). This case is passive and dissipates enough heat (even in a closed cabinet in Singapore where it's 30°) for the CPU never to throttle back when overclocked at 2Ghz (see below).

Do note that this case (which is pretty much just a massive heat sink) gets pretty hot if the RPi is running at max performance for long periods of time.

USB-SSD

Get one that fits your SSD and that ideally has Linux support. As Apple uses custom SSD connectors (prior to being soldered onto the motherboard) I had to get a converter from China. It was a bit of Russian Roulette to see if it would work or be supported on Linux. I got myself this one (chipset: Netchip Technology). As I didn't remember what type of Macbook Pro this came from, using this site to compare serial/model was useful. This USB-to-SSD converted also works on Mac and Windows by the way.

In my case, the RPi also did not provider enough power to the USB-SSD converter (although... it really should but whatevs), so be sure to use the provided power cable and plug it into a USB power source. Not doing so will cause the SSD to heat up and show a bunch of disconnects/errors in dmesg.

raspbian ~ # fdisk -l /dev/sda1 
Disk /dev/sda1: 233.8 GiB, 250999127552 bytes, 490232671 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
raspbian ~ # lsusb 
Bus 002 Device 002: ID 0525:622b Netchip Technology, Inc. 
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
raspbian ~ # lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 2: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M

eeprom update

Disconnect the USB-SSD for now.

At the time of writing we need to update the eeprom to boot from USB. I'm using the latest eeprom available to me. Note that the USB-boot eeprom is about to hit stable so you might not need to do this anymore.

There are two methods for updating. We can do it manually:

rpi-update
cd /lib/firmware/raspberrypi/bootloader/beta
rpi-eeprom-update -d -f ./pieeprom-2020-06-15.bin
# BCM2711 detected
# VL805 firmware in bootloader EEPROM
# BOOTFS /boot
# *** INSTALLING ./pieeprom-2020-06-15.bin ***
# BOOTFS /boot
# EEPROM update pending. Please reboot to apply the update.
reboot
# RPi should come back online after a reboot

Or we use rpi-eeprom-update (see article, at the bottom):

nano -w /etc/default/rpi-eeprom-update
# edit critical to stable
rpi-eeprom-update
rpi-eeprom-update -a

The good thing is that, even if you boot from a Raspbian that does not have /etc/default/rpi-eeprom-update edited to use stable instead of critical, it will not downgrade your eeprom.

Now you can plug in the SD card in an USB-SD card reader, and test if the RPi boots from USB. Note that the SD card might be slower.

All right -- so everything is working. I am keeping this SD card to update the eeprom again at a later stage (as the one we flashed is beta). If we use Archlinux or Ubuntu the eeprom update tools won't be included.

Next step is to flash Raspbian to the USB-SSD.

Boot-up from the USB-SSD.

Errors

In case you are getting an error similar to start4.elf: is not compatible you'll need to copy paste /boot/start4.elf from a Raspbian that ran rpi-update (i.e. the one from the SD card, or see below).

If you are booting (a fresh) Raspbian, it might complain about cma: Failed to reserve 256 MiB (and several other errors). The solution is running rpi-update.

Boot from the working Raspbian (using the SD card):

# check which drive is your USB-SSD (i.e. using fdisk -l or dmesg). 
# In my case I booted from USB-SD (/dev/sda) and we'll update the new/clean Raspbian on the SSD (/dev/sdb).
#
# First resize the partition, if the system never booted it'll be 1.5Gb and thus not big enough:
# Device     Boot  Start     End Sectors  Size Id Type
# /dev/sdb1         8192  532479  524288  256M  c W95 FAT32 (LBA)
# /dev/sdb2       532480 3620863 3088384  1.5G 83 Linux
fdisk /dev/sdb
# Type the following:
# p (and visually check it all makes sense)
# d
# 2
# n
# Select (default p): p
# Partition number (2-4, default 2): <enter>
# First sector (2048-490234751, default 2048): 532480 (or whichever is the same "start" from the 2nd partition) 
# Last sector, +/-sectors or +/-size{K,M,G,T,P} (532480-490234751, default 490234751): <enter>
# Created a new partition 2 of type 'Linux' and of size 233.5 GiB.
# Partition #2 contains a ext4 signature.
# Do you want to remove the signature? [Y]es/[N]o: n
# p (visually check once again it makes sense, if not you can cancel/quit by typing q)
# w (if it makes sense)
# The last command will write the changes to the partition table and sync all changes. 
# Then we need to check and resize the filesystem:
e2fsck -f /dev/sdb2
resize2fs /dev/sdb2
# If all that worked we can start mounting everything
mkdir /tmp/ssd
mount /dev/sdb2 /tmp/ssd/
mount /dev/sdb1 /tmp/ssd/boot/
mount /proc/ /tmp/ssd/proc/ -t proc
mount --rbind /sys/ /tmp/ssd/sys/
mount --rbind /dev/ /tmp/ssd/dev/
# Once everything is mounted, we're chrooting into the fresh Raspbian running on the SSD
chroot /tmp/ssd/ /bin/bash
# you can double confirm the partition size using:
df -h
# And we update the system. Again, if all this hits stable it might not be needed.
rpi-update
# say "y" when it's asking you to.
# exit the chroot and turn off the device, remove the USB-SD and leave USB-SSD connected. 
exit 
halt

My first reboot the boot process threw errors about failing to mount the root fs.

We'll need to update /etc/fstab with the correct partuuid.

# Boot from the (USB-)SD card again
# In my case sdb became sda and vice versa, so double check
lsblk
# be sure to select the right disk (the SSD, no the SD)!
mkdir /tmp/ssd
mount /dev/sda2 /tmp/ssd/
# And find the SSD here as well.
# look for the last column, partuuid, something like 
"6f6cc2fb-01"
blkid
nano -w /tmp/ssd/etc/fstab
# edit the existing partuuid's with the ones from blkid
# you'll need to edit both /boot (-01) and / (root, -02).
halt
# When rebooting from the SSD it'll go through a fsck. In my case for some reason it failed and dropped to a shell. I did a manual check and everything was fine. Rebooted and it booted normally... *shrug*

Booting

At this stage booting from the USB-SSD should work just fine. You have a working system booting from USB.

Overclocking

Last thing I'd recommend is getting a bit more juice out of your four cores.

You can quite easily overclock the RPi4 to 2Ghz (per core). It's a pretty nice boost (~25%) and worth going for. I haven't seen any heat issues leading to underclocking (throttling back), and everything runs stable. Note that under real circumstances you are unlikely to be running at 100% for extended period of times.

This guide explains how to overclock Raspbian (but the same applies for Ubuntu RPi -- I'll eventually be using Ubuntu as the OS due to its 64 bit support; at the moment Raspbian only supports a 64 bit kernel (beta) and the userland still runs 32 bit. But that'll be a follow-up article.

The gist of the article is to edit /boot/config.txt and add:

over_voltage=6
arm_freq=2000

Save the file, reboot and monitor temp (echo $((cat /sys/class/thermal/thermal_zone0/temp/1000))) and core frequency (watch -n 1 vcgencmd measure_clock arm) while running stress -c 4 to make sure the cores are running at 100%.

Resilio Sync released an update last week and on Synology these package don't auto update. Time to manually update the packages again.

On my DS1515 (more RAM, more CPU) the manual update goes by fine (stop service, manual update, browse for file, upload, start service) but my DS216j, not so much.

Attempting to upload the file instantly fails with the useful error "Operation Failed".

I tried to re-download the file (was it corrupt)... Nop. Did I select the right architecture? Yep. Decided to give it a go via via the command line. Perhaps a reboot? Nop.

First, make sure SSH is enabled and you can SSH. Oh and pro tip: add your SSH key.

Something along these lines should get you started:

# Transfer the file to the NAS
scp *.spk NAS-IP:
# Connect to the device
ssh user@NAS-IP
# Install it as root using sudo. It'll ask for your psw
sudo synopkg install *.spk
[password]
# resiliosync_armada38x_DSM6_2.7.0.1366-1.spk install successfully
# All good? Then remove the installer
rm *.spk

Somehow this worked... 🤷‍♂️ Not sure what caused the error in the first place. I am guessing the device is a bit low on RAM.

I live in Singapore. And haze is a thing. Thanks Indonesia and burning rain forests to create palm plantations. Needless to say, so far 2019 and 2020 haven't been great.

Anyhow I got myself a Novita air purifier ~2 years ago, and I stole received a Xiaomi purifier from Liyun.

The Novita is pretty dumb (the PM sensor goes randomly nuts and it needs to be quite polluted before it really kicks in. It also needs to be turned off/on again every so often for it to keep on standby — if not it goes into some sleep mode).

The manual says you should consider cleaning (aka remove dust and cat hair) every 3 or so months, and replace the filter every 6 months. It'll show a warning when the timer hits zero.

Selling new filters is how they make money I guess.

Buy a new filter (or don’t and use the same filter), reset the timer and things are back to normal.

For the Xiaomi it’s a bit harder. The “smart filter” is really just a HEPA filter with an NFC tag. It’ll calculate how often the purifier runs and calculate a % based on that. I’ve now hit 0% left on my filter after a haze season (September 2019) and ~8 months of normal usage (as it’s in the bedroom, it starts filtering around 21h00, and then runs in the quiet night mode until 9am next morning and then shuts off).

While the purifier still runs at 0%, it gives a big red warning both in the app and on the device itself and urges you to replace the filter.

You can tape off the NFC tag with some tinfoil, but that’ll just change the big red warning from “used and old filter, please replace” to “fake filter, please replace”.

Now I am not against replacing HEPA filters when they are used and are not actually filtering the air anymore. But I am also against uselessly replacing things for the sake of handing money to $BigCorp and ruining the planet with it.

Running a few tests with a relatively precise PM sensor (use this assembled kit if you are not into DIY) shows that for both my purifiers, the exhuast air is 0 PM1.0, 0 PM2.5 and 0 PM10. So that shows me that both are still working and cleaning the air just fine (further away in the room, the PM heads up to 10-20 on haze-free days).

Am I missing something here? Why throw away something that still works?

The Resilio Sync package that's provided on a Synology NAS' Packager Center is out of date (2.6.2) and has a bug that causes big files to fail to sync (and that causes a loop of endless retries).

The solution is to manually update the package from 2.6.2 to 2.6.4. Find your NAS architecture and then download (bottom of the page) the right binary and manually install it.

Note that you need to stop the Resilio Sync service running (manually stop it via Package Center) before uploading the package and installing it.

Once done, don't forget to manually start the service again.

Resilio Sync GUI will be at <NAS IP>:28888/gui/.

The howto guide to manually update the package can be found here.

I had a Nexus 5 stuck in a boot loop (Android logo/animation in a loop, not actually booting).

This is what I think I've done to fix the issue. It seemed that /persist partition was corrupt. I tried a factory reset, flash new stock images, and clear cache, etc before trying the following.

Note that I managed to boot Android 4.4, but nothing else; it did throw a shit load of errors though (Google Play crashes, etc).

First of all, get ADB & Fastboot here. You'll also need an hex editor (mac).

This will unlock your phone's OEM mode; and thus potentially voiding warranty and erasing all data (!).

Edit file paths as needed, this is a copy paste of what I can still see on my terminal.

If you know your device's WiFi MAC & Bluetooth address that'll be useful for later, as apparently that gets wiped.

Boot into recovery boot by turning off your device and then holding the power + volume down button.

$ ./fastboot-mac oem unlock
... OKAY

Flash openrecovery (TWRP):

./fastboot flash recovery ../openrecovery-twrp-2.8.5.2-hammerhead.img
sending 'recovery' (13918 KB)... OKAY
writing 'recovery'... OKAY

Boot into recovery mode (using volume buttons) from the recovery boot. ADB should work now. This will find a bunch of errors and destroy the partition.

nazgul ~/Android $ ./adb-mac shell

~ # e2fsck /dev/block/platform/msm_sdcc.1/by-name/persist
e2fsck 1.42.9 (28-Dec-2013)
Superblock has an invalid journal (inode 8).
Clear<y>? y
yes
*** ext3 journal has been deleted - filesystem is now ext2 only ***
Superblock has_journal flag is clear, but a journal inode is present.
Clear<y>? yes
/dev/block/platform/msm_sdcc.1/by-name/persist was not cleanly unmounted, check forced.
Pass 1: Checking inodes, blocks, and sizes
Journal inode is not in use, but contains data. Clear<y>?
yes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences: -(75--1098)
Fix<y>?
yes
Free blocks count wrong for group #0 (2972, counted=3996).
Fix<y>?
yes
Free blocks count wrong (2972, counted=3996).
Fix<y>?
yes
Recreate journal<y>?
yes
Creating journal (1024 blocks): Done.
*** journal has been re-created - filesystem is now ext3 again ***
/dev/block/platform/msm_sdcc.1/by-name/persist: ***** FILE SYSTEM WAS MODIFIED *****
/dev/block/platform/msm_sdcc.1/by-name/persist: 30/1024 files (3.3% non-contiguous), 1124/4096 blocks

~ # e2fsck /dev/block/platform/msm_sdcc.1/by-name/persist
e2fsck 1.42.9 (28-Dec-2013)
/dev/block/platform/msm_sdcc.1/by-name/persist: clean, 30/1024 files, 1124/4096 blocks

~ # make_ext4fs /dev/block/platform/msm_sdcc.1/by-name/persist
Creating filesystem with parameters:
Size: 16777216
Block size: 4096
Blocks per group: 32768
Inodes per group: 1024
Inode size: 256
Journal blocks: 1024
Label:
Blocks: 4096
Block groups: 1
Reserved block group size: 7
Created filesystem with 11/1024 inodes and 1102/4096 blocks
Allocating group tables: done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done

So far, so good. Persist partition was corrupt and recreated.

The original howto (see below) said root (su) was needed here; however it worked without root for me (?).

Download this file (kudos to whoever made it) and unrar it. Use your hex editor to edit last 6 digits ("00 00 00") to a valid hex value, or even better, your actual MAC address if you can remember/find it.

Now upload these two (hidden) files to /sdcard/:

nazgul ~/Android $ ./adb-mac push .bdaddr /sdcard/.bdaddr
0 KB/s (6 bytes in 0.078s)
nazgul ~/Android $ ./adb-mac push .macaddr /sdcard/.macaddr
1 KB/s (6 bytes in 0.004s)

And run these commands:

nazgul ~/Android $ ./adb-mac shell
~ # su
/sbin/sh: su: not found
~ # cd /persist
/persist # ls
/persist # mkdir bluetooth wifi
/persist # chown bluetooth:system ./bluetooth
/persist # chmod 770 ./bluetooth
/persist # ls
bluetooth
wifi
/persist # cp /sdcard/.bdaddr /persist/bluetooth
/persist # chown bluetooth:system ./bluetooth/.bdaddr
/persist # chmod 660 ./bluetooth/.bdaddr
/persist # chown wifi:system ./wifi
/persist # chmod 770 ./wifi
/persist # cp /sdcard/.macaddr /persist/wifi
/persist # chown wifi:system ./wifi/.macaddr
/persist # chmod 660 ./wifi/.macaddr
/persist # rm /sdcard/.bdaddr
/persist # rm /sdcard/.macaddr
/persist # reboot

Go back into recovery boot and flash Android (I flashed 4.4 first, made sure it worked, and then flashed 6.0.1 (latest at this time); but you can probably flash latest version right away. Also unzip the zip file with all the images inside the .tar.gz -- we'll need the files later:

nazgul ~/Downloads/hammerhead-mmb29k.6 $ ./flash-all.sh
sending 'bootloader' (3120 KB)... OKAY
writing 'bootloader'... OKAY
rebooting into bootloader... OKAY
sending 'radio' (45425 KB)... OKAY
writing 'radio'... OKAY
rebooting into bootloader... OKAY
archive does not contain 'boot.sig'
archive does not contain 'recovery.sig'
archive does not contain 'system.sig'
--------------------------------------------
Bootloader Version...: HHZ12k
Baseband Version.....: M8974A-2.0.50.2.28
Serial Number........: 0644c9920b105eb5
--------------------------------------------
checking product... OKAY
checking version-bootloader... OKAY
checking version-baseband... OKAY
sending 'boot' (9154 KB)... OKAY
writing 'boot'... OKAY
sending 'recovery' (10012 KB)... OKAY
writing 'recovery'... OKAY
sending 'system' (1020405 KB)... OKAY
writing 'system'... OKAY
erasing 'userdata'... OKAY
erasing 'cache'... OKAY
rebooting...

Note that it won't actually boot yet, so go back into recovery boot, and flash userdata and cache (not sure why they go missing or get entirely erased):

nazgul ~/Downloads/hammerhead-mmb29k.6 $ ./fastboot flash userdata image-hammerhead-mmb29k/userdata.img
sending 'userdata' (137318 KB)... OKAY
writing 'userdata'... OKAY
nazgul ~/Downloads/hammerhead-mmb29k.6 $ ./fastboot flash cache image-hammerhead-mmb29k/cache.img
sending 'cache' (13348 KB)... OKAY
writing 'cache'... OKAY

Execute a normal boot now, and wait 5 to 10 minutes.

Android should boot up normally now.

You can also OEM lock your phone again, if you wish (but a sticky bit has been set).

I've followed these forum posts.