Skip to content

Creation of subfolders using desktop client after a move across shares can lead to duplicated folders. #28018

Closed
#28022
Closed
Creation of subfolders using desktop client after a move across shares can lead to duplicated folders.#28018
#28022
Assignees
PVince81mrow4a
Labels
Type:Bugsev2-highstatus/STALE
Milestone
10.0.3

Description

@SergioBertolinSG
SergioBertolinSG
opened on May 26, 2017

Steps to reproduce

  1. As admin create user0 and user1.
  2. As user0 creates folderA and folderB.
  3. As user0 share folderA and folderB with user1.
    Connect the desktop client using user1 and do:
  4. Create folder 'one' in folderA.
  5. Create folder 'two' in folderA/one.
  6. Move 'folderA/one' subfolder to folderB.
  7. Create folder 'three' in 'folderB/one/two' .
  8. Access using the web ui using user0 or user1 and enter the folder folderB/one.

Expected behaviour

There is a folder 'two' inside 'folderB/one'.

Actual behaviour

Folder called 'two' is duplicated inside 'folderB/one' with the same name.

Server configuration

Operating system:
Ubuntu 14.04

Web server:
Apache

Database:
MySQL

PHP version:
7.0

ownCloud version: (see ownCloud admin page)
current master
{"installed":"true","maintenance":"false","needsDbUpgrade":"false","version":"10.0.2.0","versionstring":"10.0.2 RC1","edition":"Community","productname":"ownCloud"}

Updated from an older ownCloud or fresh install:
Fresh

The content of config/config.php:

Are you using external storage, if yes which one: local/smb/sftp/...
No.

Are you using encryption:
No

Logs

Client configuration

Browser
Chrome

Logs

cc @SamuAlfageme

Activity

  1. SergioBertolinSG
    added
    Type:Bug
    sev2-high
    on May 26, 2017
  2. SergioBertolinSG
    added this to the triage milestone on May 26, 2017
  3. SergioBertolinSG
    changed the title [-]Duplication of folders while using dektop client when moving a folder across shares and creating new subfolders inside existing subfolders.[/-] [+]Duplication of folders while using desktop client when moving a folder across shares and creating new subfolders inside existing subfolders.[/+] on May 26, 2017
  4. SergioBertolinSG
    changed the title [-]Duplication of folders while using desktop client when moving a folder across shares and creating new subfolders inside existing subfolders.[/-] [+]Creation of subfolders using desktop client after a move across shares can lead to duplicated folders.[/+] on May 26, 2017

  5. PVince81 commented on May 26, 2017

    @PVince81
    PVince81
    on May 26, 2017
    Contributor

    No need for desktop client. Just move using cadaver.

    After step 6 I see this in the DB:

    +---------+--------+--------+--------------------------------------------+----------------------------------+-------+
| storage | fileid | parent | path                                       | etag                             | size  |
+---------+--------+--------+--------------------------------------------+----------------------------------+-------+
|       3 |     34 |     28 | files/folderA                              | 59281e68589d6                    |     0 |
|       3 |     41 |     40 | files/folderA/one/two                      | 59281e2a7dada                    |     0 |
|       3 |     35 |     28 | files/folderB                              | 59281e685ba2c                    |     0 |
|       3 |     40 |     35 | files/folderB/one                          | 59281e2a82e38                    |     0 |
|       3 |     48 |     40 | files/folderB/one/two                      | 59281e70d05f9                    |     0 |

    Looks like after moving "one" from the source the filecache entry is still there.

    After step 7:

    |       3 |     34 |     28 | files/folderA                              | 59281e68589d6                    |     0 |
|       3 |     41 |     40 | files/folderA/one/two                      | 59281e2a7dada                    |     0 |
|       3 |     35 |     28 | files/folderB                              | 59281ee3c1427                    |     0 |
|       3 |     40 |     35 | files/folderB/one                          | 59281ee3c1427                    |     0 |
|       3 |     48 |     40 | files/folderB/one/two                      | 59281ee3c1427                    |     0 |
|       3 |     49 |     48 | files/folderB/one/two/three                | 59281ee3bf0c7                    |     0 |

    On disk:

    user0/files
user0/files/folderB
user0/files/folderB/one
user0/files/folderB/one/two
user0/files/folderB/one/two/three
user0/files/folderA

    So we have:

    • filecache inconsistency
    • some weird response in the web UI causing two "two" folders even though the filecache puts them in two separate folders

    Running occ files:scan --all does not fix the issue 😢

  6. PVince81
    modified the milestones: 10.0.3, triage on May 26, 2017

  7. PVince81 commented on May 26, 2017

    @PVince81
    PVince81
    on May 26, 2017
    Contributor

    might also happen when moving between external storage mount points

  8. PVince81 commented on May 26, 2017

    @PVince81
    PVince81
    on May 26, 2017
    Contributor

    Hmm no, might be specific to shared storages.

    From what I see it goes to Updater::renameFromStorage() but fails to recognize that both come from the same wrapped storage (the one from user0). Inspecting shows that both have a different stored mount point which makes sense.

    So this triggers the logic of $this->cache->moveFromCache which might be already. But there the paths look strange:

    $sourceId                        = (int) '56';
$sourceLength                    = (int) '4';
$sourcePath                      = (string[4]) 'what';
$sourceStorageId                 = (int) '3';
$targetPath                      = (string[18]) 'files/folderB/what';
$targetStorageId                 = (int) '3';

    The $targetPath is absolute but $sourcePath is relative. That could be the issue...

  9. PVince81 commented on May 26, 2017

    @PVince81
    PVince81
    on May 26, 2017
    Contributor

    Just tried with moving from received shared folder to regular root, one of the paths is also absolute.
    That must be because we're moving from a shared storage to home storage.

    In the case of shared storage A to shared storage B, one of them is resolved to the home storage but not the other one.

  10. PVince81 commented on May 26, 2017

    @PVince81
    PVince81
    on May 26, 2017
    Contributor

    Ok, I'm getting closer. In the cross-shared-storage move, both paths must be:
    $sourcePath='files/folderA/what and $targetPath='files/folderB/what'. Without this the SQL command cannot find all children within "folderA" so it doesn't move the children.

    Now this is likely due to having one of the shared storages which was resolved back the to the home storage but not the other one. Or maybe the cache instance is not compatible with the storage.

    Needs further research.

  11. PVince81 commented on May 26, 2017

    @PVince81
    PVince81
    on May 26, 2017
    Contributor

    Fix is here #28022

  12. PVince81
    self-assigned this
    on May 26, 2017

  13. PVince81 commented on May 26, 2017

    @PVince81
    PVince81
    on May 26, 2017
    Contributor

    @SergioBertolinSG by the way, the cadaver thing I tested against the old dav endpoint.

    From the fix it's not related to any specific dav endpoint as it's a core FS thing.

  14. 3 remaining items

  15. PVince81 commented on Jun 26, 2017

    @PVince81
    PVince81
    on Jun 26, 2017
    Contributor

    Fix still pending review #28022

  16. PVince81 commented on Jun 26, 2017

    @PVince81
    PVince81
    on Jun 26, 2017
    Contributor

    To fix the filecache, we likely need something that repairs parent-child relationships based on the path. Or the other way around: #22866 (comment)

  17. PVince81 commented on Jun 26, 2017

    @PVince81
    PVince81
    on Jun 26, 2017
    Contributor

    This query helps find out whether parent-child relationships are broken. In the case above:

    select fc.fileid, fc.path, fc2.fileid, fc2.path from oc_filecache fc, oc_filecache fc2 where fc.parent = fc2.fileid and CONCAT(fc2.path, '/', fc.name) != fc.path and fc2.path != '';
+--------+-----------------------+--------+-------------------+
| fileid | path                  | fileid | path              |
+--------+-----------------------+--------+-------------------+
|     31 | files/folderA/one/two |     30 | files/folderB/one |
+--------+-----------------------+--------+-------------------+

  18. PVince81 commented on Jun 26, 2017

    @PVince81
    PVince81
    on Jun 26, 2017
    Contributor

    this can be fixed manually by changing the path of the entry "31" to the one of the parent, so renaming "folderA" to "folderB":
    update oc_filecache set path='files/folderB/one/two', path_hash=md5(path) where fileid=31;

    EDIT: ⚠️ I forgot to add the path_hash update, it's there now

  19. PVince81 commented on Jun 26, 2017

    @PVince81
    PVince81
    on Jun 26, 2017
    Contributor

    Tried to find a way to update all such entries with a single SQL statement but can't because an UPDATE operation cannot select stuff at the same time on the same time in a subquery: https://stackoverflow.com/a/22378467

    So the best would be to have PHP code that iterates over all the entries and repairs them. Additionally it would need to check whether the target path/entry doesn't already exist.

  20. PVince81 commented on Jun 26, 2017

    @PVince81
    PVince81
    on Jun 26, 2017
    Contributor

    Experimental repair routine that runs during files scan: #28217 OBSOLETE

  21. PVince81
    closed this as completedin #28022on Jun 26, 2017

  22. PVince81 commented on Jun 28, 2017

    @PVince81
    PVince81
    on Jun 28, 2017
    Contributor

    Another attempt at a better repair routine: #28253

  23. PVince81 commented on Jun 28, 2017

    @PVince81
    PVince81
    on Jun 28, 2017
    Contributor

    For the record: this is also reproducible if both shares come from two different users.
    In that case the storage value will also be wrong in oc_filecache for the entry that has the wrong path.

  24. PVince81 commented on Jun 30, 2017

    @PVince81
    PVince81
    on Jun 30, 2017
    Contributor

    Apparently also reproducible by simply moving a folder that has subfolders out of a received share:

    1. user0 shares "folderA" with user1
    2. user1 creates "folderA/one/two/three"
    3. user1 moves "folderA/one" to "one" which moves it out 💥 . filecache inconsistency
    4. user1 moves "one" back into "folderA": getting 403 "target already exists". Target is not visible in listing but you can "cd" into it.

  25. PVince81 commented on Jul 3, 2017

    @PVince81
    PVince81
    on Jul 3, 2017
    Contributor

    Observed in v10.0.2, v9.1.4, v9.1.0, v9.0.10, v9.0.0.

    Does not occur on v8.2.11.

    Likely related to recent shared storage related refactorings.

  26. PVince81 commented on Jul 3, 2017

    @PVince81
    PVince81
    on Jul 3, 2017
    Contributor

    Repair step coming close to completion: #28253

  27. ownclouders
    added
    status/STALE
    on Dec 21, 2017

  28. lock commented on Aug 1, 2019

    @lock
    lockBot
    on Aug 1, 2019

    This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

  29. lock
    locked as resolved and limited conversation to collaborators on Aug 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type:Bugsev2-highstatus/STALE

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions