FileUploadCompletionNotification not working with Iothub GWV2

Context

OS and version used: Windows 10
Java runtime used: 17.0.4
SDK version used: 2.4.0

Description of the issue

Can not send FileUploadCompletionNotification. IotHubClientException with BadFormat is thrown.
As i used the FileUploadSample as a basis this should be reproducable relativly easily i hope.

Code sample exhibiting the issue

Slightly adapted FileUploadSample from the samples.

public class FileUploadSample
{
    private static final List<String> fileNameList = new ArrayList<>();

    /**
     * Upload file or directories to blobs using IoT Hub.
     *
     * @param args 
     * args[0] = IoT Hub connection string
     * args[1] = File or directory to upload
     */
    public static void main(String[] args)
            throws IOException, URISyntaxException
    {
        String connString = "XXX";

        System.out.println("Starting...");
        System.out.println("Beginning setup.");

        // File upload will always use HTTPS, DeviceClient will use this protocol only
        //   for the other services like Device Telemetry, Device Method and Device Twin.
        IotHubClientProtocol protocol = IotHubClientProtocol.MQTT;

        System.out.println("Successfully read input parameters.");
        System.out.format("Using communication protocol %s.\n",
                protocol.name());

        DeviceClient client = new DeviceClient(connString, protocol);

        System.out.println("Successfully created an IoT Hub client.");
        
        try
        {
            FileUploadSasUriResponse sasUriResponse = client.getFileUploadSasUri(new FileUploadSasUriRequest("JustSomeFile"));

            FileUploadCompletionNotification completionNotification = new FileUploadCompletionNotification(sasUriResponse.getCorrelationId(), false);
            client.completeFileUpload(completionNotification);
        }
        catch (IotHubClientException e) 
        {
            System.out.println("IotHubClientException: " + e.getStatusCode());
        }
        catch (Exception e)
        {
            System.out.println("On exception, shutting down \n" + " Cause: " + e.getCause() + " \nERROR: " +  e.getMessage());
            System.out.println("Shutting down...");
            client.close();
        }
        client.close();
        
    }

}

Console log of the issue

Starting...
Beginning setup.
Successfully read input parameters.
Using communication protocol MQTT.
2023-12-11 19:54:10,169 INFO (main) [com.microsoft.azure.sdk.iot.device.transport.ExponentialBackoffWithJitter] - NOTE: A new instance of ExponentialBackoffWithJitter has been created with the following properties. Retry Count: 2147483647, Min Backoff Interval: 100, Max Backoff Interval: 10000, Max Time Between Retries: 100, Fast Retry Enabled: true
2023-12-11 19:54:10,233 DEBUG (main) [com.microsoft.azure.sdk.iot.device.ClientConfiguration] - Device configured to use software based SAS authentication provider
2023-12-11 19:54:10,249 DEBUG (main) [com.microsoft.azure.sdk.iot.device.DeviceClient] - Initialized a DeviceClient instance using SDK version 2.4.0
Successfully created an IoT Hub client.
IotHubClientException: BAD_FORMAT
2023-12-11 19:54:10,593 INFO (main) [com.microsoft.azure.sdk.iot.device.DeviceClient] - Closing device client...
2023-12-11 19:54:10,594 INFO (main) [com.microsoft.azure.sdk.iot.device.DeviceClient] - Device client closed successfully
Consider setting the DEBUG environment variable to '*'. This will produce a much more verbose output that will help debugging
Don't forget to remove any connection string information!

I tried the same with a different iothub instance which is working as expected.
I could see the iothub i used for the test has the GWV2 feature enabled. This may be the cause for the Problem but i can not confirm it as it is not possible to change this feature.

Is there an actual file being uploaded when you repro this? Sometimes, the BAD_FORMAT error just indicates that IoT hub doesn't have a record of that SAS URI yet. It may take a second or two between requesting one and it being completable and normally that time is filled by uploading a file.

Adding a Thread.sleep(2000) between receiving the SAS URI and completing it could be another way to check if this is just a timing issue.

I think BAD_FORMAT is a bad error message in this case.
This is just a minimal sample i found the Problem in a more complex Szenario with a file upload.
I have run the sample with a sleep time of 20000 ms and it comes to the same result.

I think BAD_FORMAT is a bad error message in this case.

Yep, this is something we have brought up to the service team before, but we can't change this behavior now, unfortunately.

How reliably do you see this behavior repro? 100% of the time?

I can see this behavior every time until i hit the limit of too much active downloads.

I think it would be nice to confirm the issue with another sdk or with a simple http client but I did not have the time yet.

This is likely a question of service behavior, so we'll try to investigate this when we have the time

We ran into exactly the same issue with a hub on GWV2. The issue seems to be that the FileUploadCompletionNotification(String, Boolean) constructor does not fill in statusCode and statusDescription correctly. When they remain null, the GWV2 refuses our request with validation error (a BAD_FORMAT error message). If we set those fields explicitly using a different constructor, releasing the URI works.

I think, this additional validation is something that needs to be changed in GWV2, as it breaks existing clients that cannot be easily updated to workaround this new validation.

Navigation Menu

Repository navigation

FileUploadCompletionNotification not working with Iothub GWV2 #1771

Context

Description of the issue

Code sample exhibiting the issue

Console log of the issue

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

FileUploadCompletionNotification not working with Iothub GWV2 #1771

Description

Context

Description of the issue

Code sample exhibiting the issue

Console log of the issue

Activity

Robsmon commented on Dec 11, 2023

timtay-microsoft commented on Dec 12, 2023

timtay-microsoft commented on Dec 12, 2023

Robsmon commented on Dec 12, 2023

timtay-microsoft commented on Dec 12, 2023

Robsmon commented on Dec 12, 2023

Robsmon commented on Dec 12, 2023

timtay-microsoft commented on Dec 12, 2023

mischak commented on Feb 13, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions