Not recognizing custom characteristic/service

[r00li]

Hi!
So, I am very new to Openhab (just installed it yesterday on my RPI 0W) and I am having some issues getting this binding to work with my custom devices.

So I have a few custom BLE devices (built by me) - powered by either a Microchip BLE module or the nordic nrf51822. Temperature sensors, blind controllers, switches and dimmers basically. I can get my devices to connect just fine, but I can't seem to get my own characteristics to show up.

For instance this is my device when connected (I have enabled for unknown channels to display as binary):

This is my switch module. It has a single characteristic - you write 255 to turn the light on and you write 0 to turn the light off. It has read/write/notify properties.

This is my service xml file that I added:

<Service xsi:noNamespaceSchemaLocation="http://schemas.bluetooth.org/Documents/service.xsd"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" name="RSwitch V1 service"
         type="com.itag.service.simple_button" uuid="6a741280-71d6-11e5-952a-0002a5d5c51b" last-modified="2018-02-22">
<InformativeText>
    <Abstract>
        RSwitch V1 bluetooth service
    </Abstract>
    <Summary>
        RSwitch V1 bluetooth service
    </Summary>
</InformativeText>
<Dependencies>
    <Dependency>This service has no dependencies on other GATT-based services.</Dependency>
</Dependencies>
<GATTRequirements>
    <Requirement subProcedure="Write Characteristic Value">Mandatory</Requirement>
    <Requirement subProcedure="Notification">Mandatory</Requirement>
    <Requirement subProcedure="Read Characteristic Descriptors">Mandatory</Requirement>
    <Requirement subProcedure="Write Characteristic Descriptors">Mandatory</Requirement>
</GATTRequirements>
<Transports>
    <Classic>false</Classic>
    <LowEnergy>true</LowEnergy>
</Transports>
<ErrorCodes>
</ErrorCodes>
<Characteristics>
    <Characteristic name="RSwitch_V1"
                    type="com.r00li.bluetooth.characteristic.rswitchv1">
        <InformativeText>Button
        </InformativeText>
        <Requirement>Mandatory</Requirement>
        <Properties>
            <Read>Mandatory</Read>
            <Write>Mandatory</Write>
            <WriteWithoutResponse>Excluded</WriteWithoutResponse>
            <SignedWrite>Excluded</SignedWrite>
            <ReliableWrite>Excluded</ReliableWrite>
            <Notify>Mandatory</Notify>
            <Indicate>Excluded</Indicate>
            <WritableAuxiliaries>Excluded</WritableAuxiliaries>
            <Broadcast>Excluded</Broadcast>
        </Properties>
    </Characteristic>
</Characteristics>
</Service>

And this is my characteristic:

<?xml version="1.0" encoding="UTF-8"?>
<Characteristic xsi:noNamespaceSchemaLocation="http://schemas.bluetooth.org/Documents/characteristic.xsd"
                xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" name="RSwitch_V1"
                type="com.r00li.bluetooth.characteristic.rswitchv1" uuid="9ed90c00-71d7-11e5-977a-0002a5d5c51b" last-modified="2018-11-10"
                approved="No">
    <Value>
        <Field name="Switch">
            <Format>uint8</Format>
            <InformativeText>Switch status</InformativeText>
            <Enumerations>
                <Enumeration key="0" value="Off"/>
                <Enumeration key="255" value="On"/>
            </Enumerations>
        </Field>
    </Value>
</Characteristic>

To be honest, I am not entirely sure if what I am doing is correct, but going from examples this is what I came up with. Now I think that the binding loads my xmls correctly - the logs show that it was complaining when I had an error in the XML, and it's not complaining now.

One other thing that I find weird is that changing the value for the binary characteristic from that UI does nothing. At least it doesn't look like it does anything - the light doesn't turn on/off.

[vkolotov]

Hi @r00li, it is great that you built your own devices, this is effectively what I'm trying to do as well. I'll be very keen on helping you with this.

First thing to check is file names. How do you name them? Should be the same as you put in the "type" tag, e.g.:

com.itag.service.simple_button.xml 
com.r00li.bluetooth.characteristic.rswitchv1.xml

I also suggest to change service name to something like that:

com.r00li.bluetooth.service.rswitch.xml

Also, you must remove and add your device in OH when you put your custom gatt specs. Basically, if a binary channel is already added, then it will not replaced with "user" channels from gatt files. So you have to remove your "thing", change "GATT services and characteristics parsing strategy" to "Recognised only" or "Recognised + system ..." and then add your thing again.

In theory this should work. However, there is something you could improve in the design of your GATT structure.
There is a fundamental design consideration that you might like to take into account. The thing is that, even if you enable a flag (send 255) this would not 100% mean that your device switches its state, there are lots of things can happen preventing it to switch its state. Therefore, from my point of view there should be at least two characteristics:

• One for changing state
• One for reporting state

The one that reports its state should support BLE "notifications", e.g. it should notify clients when its state changes. The one that changes state, effectively it is a "command" characteristic.

If you look at the way how standard characteristics are designed, you may notice that if a BLE device supports changing its state, it has a special characteristic called "control point", for example: https://github.com/sputnikdev/bluetooth-gatt-parser/blob/master/src/main/resources/gatt/characteristic/org.bluetooth.characteristic.heart_rate_control_point.xml

There are lots of similar examples in the standard characteristics like that. Effectively they encourage people to use "control point" as a single entry for changing device state and multiple characteristics for reading device states. It really makes sense if you think about it.

I've got some good examples that I use to control some switches at my home, unfortunately I don't have them handy now, but I'll attach it here once I'm at home.

[r00li]

Hi...
Excellent. I do plan on open sourcing these as soon as I have a bit of time to document everything properly. And maybe replace a bunch of capacitors and inductors with a single chip balun on the NRF51 - soldering 0402 components by hand is a pain... Anyways... to the issue.

Yeah apparently I did make a mistake while adding a type. But that is just because I copied and pasted the example files so many times that I apparently at one point forgot to change the type. I tried it again with the correct type - sadly it didn't make a difference. The service and characteristic are named v1 because there is also a v2 which supports dimming.

And yeah, I know that I need to remove and add a thing when I change the specs (I also change the path to the specs files to something random and then back to correct path again when changing those files - just to force it to reload them). My parsing strategy is set to "Recognised + system..."
.

As for the GATT style... I do have separate channels for control and status on one of my other devices. These were the first devices that I made and they have a really simple characteristic set. Besides - it was never an issue (and I've been using these switches for two years now). Currently if you set the value to anything other than something valid the V2 switch (based on NRF51, V1 is based on the Microchip BLE module) will just discard the value and set the correct one again. If I write [ff] to the binary characteristic via openhab (which should turn the light on), the switch just resets it back to [00]. The Microchip module is slightly more weird though. At least for these modules this will stay as is - mainly because they are wired into my electrical panel and I don't really want to fish them out. But I will take this into consideration for the future.

[vkolotov]

Hi @r00li, please find an example here: https://www.dropbox.com/s/383i56s5ilbx3yx/bluetooth_smart.zip?dl=0

This is an example of gatt files for a smart lock. I believe it is a self explanatory. Please try to adapt it for your switch.

[r00li]

Tried it again... still no luck...

com.r00li.bluetooth.service.rswitchv1.xml:

<?xml version="1.0" encoding="utf-8"?><!-- Copyright 2011 Bluetooth SIG, Inc. All rights reserved. -->
<Service xsi:noNamespaceSchemaLocation="http://schemas.bluetooth.org/Documents/service.xsd"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" type="com.r00li.bluetooth.service.rswitchv1" uuid="6a741280-71d6-11e5-952a-0002a5d5c51b"
         name="RSwitch V1">
    <InformativeText>
        <Abstract>
            Switch service
        </Abstract>
        <Summary>
            Switch service
        </Summary>
    </InformativeText>
    <Dependencies>
        <Dependency>This service is not dependent upon any other services.</Dependency>
    </Dependencies>
    <GATTRequirements>
        <Requirement subProcedure="Write Characteristic Value">Mandatory</Requirement>
        <Requirement subProcedure="Notifications">Mandatory</Requirement>
        <Requirement subProcedure="Read Characteristic Descriptors">Mandatory</Requirement>
    </GATTRequirements>
    <Transports>
        <Classic>false</Classic>
        <LowEnergy>true</LowEnergy>
    </Transports>
    <ErrorCodes>
    </ErrorCodes>
    <Characteristics>
        <Characteristic name="RSwitch_V1" type="com.r00li.bluetooth.characteristic.rswitchv1">
            <InformativeText>
                Switch state
            </InformativeText>
            <Requirement>Mandatory</Requirement>
            <Properties>
                <Read>Mandatory</Read>
                <Write>Mandatory</Write>
                <WriteWithoutResponse>Excluded</WriteWithoutResponse>
                <SignedWrite>Excluded</SignedWrite>
                <ReliableWrite>Excluded</ReliableWrite>
                <Notify>Mandatory</Notify>
                <Indicate>Excluded</Indicate>
                <WritableAuxiliaries>Excluded</WritableAuxiliaries>
                <Broadcast>Excluded</Broadcast>
            </Properties>
            <Descriptors>
                <Descriptor name="Client Characteristic Configuration"
                            type="org.bluetooth.descriptor.gatt.client_characteristic_configuration">
                    <Requirement>Mandatory</Requirement>
                    <Properties>
                        <Read>Mandatory</Read>
                    </Properties>
                </Descriptor>
            </Descriptors>
        </Characteristic>
    </Characteristics>
</Service>

com.r00li.bluetooth.characteristic.rswitchv1.xml:

<?xml version="1.0" encoding="UTF-8"?><!-- Copyright 2011 Bluetooth SIG, Inc. All rights reserved. -->
<Characteristic xsi:noNamespaceSchemaLocation="http://schemas.bluetooth.org/Documents/characteristic.xsd"
                xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                type="com.r00li.bluetooth.characteristic.rswitchv1" name="RSwitch_V1"
                uuid="9ED90C00-71D7-11E5-977A-0002A5D5C51B">
    <InformativeText></InformativeText>
    <Value>
        <Field name="Switch state">
            <Requirement>Mandatory</Requirement>
            <Format>8bit</Format>
            <Enumerations>
                <Enumeration key="0" value="Off"/>
                <Enumeration key="255" value="On"/>
                <ReservedForFutureUse start="1" end="254"/>
            </Enumerations>
        </Field>
    </Value>
</Characteristic>

One thing that I've noticed is that none of these examples that I've seen use full 128 bit UUID for services/characteristics. Could be anything to do with that? Another thing that I've noticed is that this service also includes the Client Characteristic Configuration descriptor. Should this be included? Also... what does Excluded mean when configuring service - that it must not have this option or that it doesn't matter if it has this option or not?

Here is what nRF connect says about my device:

[vkolotov]

Right.

One thing that I've noticed is that none of these examples that I've seen use full 128 bit UUID for services/characteristics.

Could you please try to use the very first bit of your UUIDs? E.g.:

<Characteristic xsi:noNamespaceSchemaLocation="http://schemas.bluetooth.org/Documents/characteristic.xsd"
                xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                type="com.r00li.bluetooth.characteristic.rswitchv1" name="RSwitch_V1"
                uuid="9ED90C00">

<Service xsi:noNamespaceSchemaLocation="http://schemas.bluetooth.org/Documents/service.xsd"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" type="com.r00li.bluetooth.service.rswitchv1" uuid="6a741280"
         name="RSwitch V1">

This just reminds me something... There is probably a bug matching UUIDs of real characteristics with GATT definitions. Worthwhile to try before we dig further.

[r00li]

Yes! This worked. My characteristic showed up correctly now.

And I could also turn the light on by choosing On from the drop down. Strangely enough I can't seem to turn it off by selecting Off. I don't see any errors in the log (at least not one that would be obvious), but the light just won't turn off. What is weird here is that if I connect to it with my phone I now have to write 16 bit 0 for it to turn off. But that only happens when I modify the value using this binding. If I change it with any of my phones or my Mac or a python BlueZ library on the Pi I just write 8 bits. So something is still a bit iffy here. I have to test my V2 module with the nrf chip.

[vkolotov]

Good, what is the type of the characteristic? Make sure you specify a correct on, quite important. Is it a signed/unsigned number?

[vkolotov]

Here is a list of supported types: https://www.bluetooth.com/specifications/assigned-numbers/format-types

play with uint8 and sint8

[vkolotov]

I assume that it is unsigned 8 bit byte, if you write 255 value to turn it on (all bits are 1), so try uint8.

[r00li]

It is uint8. I've tried uint8, 8bit and I've also tried uint16 just to see what happens. With uint16 I get errors from the binding that there was an issue while reading.

[vkolotov]

Are you saying that uint8 does not work?

[r00li]

Yes. That is exactly what I am saying.

[vkolotov]

Ok, give me some time to get a new unittest for this.

[r00li]

Ok. I will try it with my V2 switch as well when I come home to see if it behaves any differently. And I'll try connecting a debugger to it if I have some time.

[vkolotov]

Hey @r00li, is this what you would expect?
https://github.com/sputnikdev/bluetooth-gatt-parser/blob/31506f47638fb3203ed68cffdd0c1f063a2a37ab/src/test/java/org/sputnikdev/bluetooth/gattparser/GenericCharacteristicParserIntegrationTest.java#L438

assertEquals("11111111", Integer.toBinaryString(Byte.toUnsignedInt(raw[0])));

In other words your 8bit characteristic gets written with all 1.