Skip to content

Rounding when two-bytes data is converted to one-byte data? #125

Open

Description

@cjunekim

It seems to truncate the value when converting two-bytes to one-byte data.

For example, pitchbend to cc. Suppose the pitchbend data incoming is 0x60 0x68 (LSB and MSB, in order). Its normalized value is(tested on Lua console):

> ((0x68 << 7) | 0x60&0x7f)/16383
0.81840932674113

When it is handed to CC, it is converted as,

> ((0x68 << 7) | 0x60&0x7f)/16383 * 127.0
103.93798449612

This value is handed to an unsigned int and the fractional part is discarded becoming 103, which, in turn, in hex is 0x67. It isn't 0x68, even though it was closer to 0x68, mathematically. It's because there is no rounding algorithm when a conversion from higher resolution to lower resolution occurs. I think this can cause some confusion and issues.

What do you think?

Activity

  1. cbdevnet commented on Jun 24, 2024

    @cbdevnet
    Owner

    Hm yes, not optimal in this case but hard to fix. Will think about it some more..

  2. added
    enhancementImprovements on existing features
    help wantedIssues where contributions are appreciated a great deal
    documentationIssues within the documentation
    on Jun 24, 2024
  3. self-assigned this
    on Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

core wishlistNew core feature requestsdocumentationIssues within the documentationenhancementImprovements on existing featureshelp wantedIssues where contributions are appreciated a great deal

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions