Repository navigation
Specifying nested file trees without IDs in the API #1073
Description
Nature of issue?
- Existing feature enhancement
Feature enhancement details:
As part of the Sketch upload API work, we want a nicer way of specifying file trees.
Below, "file" means file or directory, as they're mostly the same for this discussion.
Current approach
The current API shape closely mirrors the shape of the data stored in mongodb:
- requires BSON-object ids to be generated by clients
- files are specified as a flat array of all files
- hierarchy/nesting is defined by the file IDs in a
childrenarray on the relevant object
Example
{
"name": "Summer accordion",
"updatedAt": "",
"isSaving": false,
"files": [
{
"name": "root",
"id": "5cd98108d498ade50d793853",
"_id": "5cd98108d498ade50d793853",
"children": [
"5cd98108d498ade50d793850",
"5cd98108d498ade50d793851",
"5cd98108d498ade50d793852",
"5cd9810dd498ade50d793854"
],
"fileType": "folder",
"content": "",
"isSelectedFile": false
},
{
"name": "sketch.js",
"content": "function setup() {\n createCanvas(400, 400);\n}\n\nfunction draw() {\n background(220);\n}",
"id": "5cd98108d498ade50d793850",
"_id": "5cd98108d498ade50d793850",
"isSelectedFile": false,
"fileType": "file",
"children": []
},
{
"name": "index.html",
"content": "<!DOCTYPE html>\n<html>\n <head>\n <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/p5.min.js\"></script>\n <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/addons/p5.dom.min.js\"></script>\n <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/addons/p5.sound.min.js\"></script>\n <link rel=\"stylesheet\" type=\"text/css\" href=\"style.css\">\n <meta charset=\"utf-8\" />\n\n </head>\n <body>\n <script src=\"sketch.js\"></script>\n </body>\n</html>\n",
"id": "5cd98108d498ade50d793851",
"_id": "5cd98108d498ade50d793851",
"fileType": "file",
"children": [],
"isSelectedFile": false
},
{
"name": "style.css",
"content": "html, body {\n margin: 0;\n padding: 0;\n}\ncanvas {\n display: block;\n}\n",
"id": "5cd98108d498ade50d793852",
"_id": "5cd98108d498ade50d793852",
"fileType": "file",
"children": [],
"isSelectedFile": false
},
{
"name": "my-dir",
"id": "5cd9810dd498ade50d793854",
"_id": "5cd9810dd498ade50d793854",
"content": "",
"children": ["5cd98167d498ade50d793858"],
"fileType": "folder",
"isFolderClosed": false,
"isSelectedFile": true
},
{
"name": "something.js",
"id": "5cd98167d498ade50d793858",
"_id": "5cd98167d498ade50d793858",
"content": "var answer = 42;",
"children": [],
"fileType": "file"
}
]
}
New approach
Files can be specified as a JSON object, without IDs using just the filename. The name is used to enforce the uniqueness of a file in a directory, and directories support nesting.
The server would be responsible for creating IDs and converting to the structure required by mongodb.
Example
{
"name": "Summer accordion",
"files": {
"sketch.js": {
"content": "function setup() {\n createCanvas(400, 400);\n}\n\nfunction draw() {\n background(220);\n}"
},
"index.html": {
"content": "<!DOCTYPE html>\n<html>\n <head>\n <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/p5.min.js\"></script>\n <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/addons/p5.dom.min.js\"></script>\n <script src=\"https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/addons/p5.sound.min.js\"></script>\n <link rel=\"stylesheet\" type=\"text/css\" href=\"style.css\">\n <meta charset=\"utf-8\" />\n\n </head>\n <body>\n <script src=\"sketch.js\"></script>\n </body>\n</html>\n"
},
"style.css": {
"content": "html, body {\n margin: 0;\n padding: 0;\n}\ncanvas {\n display: block;\n}\n"
},
"my-dir": {
"children": {
"something.js": {
"content": "var answer = 42;"
}
}
}
}
}Open questions
- Do we need to limit how nested the files can be?
- How do binary files get uploaded? I think this format doesn't handle it, and there's an endpoint to upload individual binary files.
Activity
catarak commentedon May 15, 2019
Overall, this looks go to me! To answer your questions:
Right now, there is no limit; however, one data point for this is that the Sidebar styling breaks if a file is more than four levels nested, see _sidebar.scss. Given that this hasn't come up, ever, I think it makes sense for the sake of simplicity to limit the level of nesting. For the relatively small sketches that folks will upload to the web editor, I don't think they'll need more than four or five levels of nesting, anyway.
My instinct is that this should be a separate endpoint—a user could upload a file or a folder of files, and get back the S3 URL, or some other ID, and then use to build the sketch JSON. We could also allow users to use their own media hosting, e.g. they can use whatever media url they want as long as it's properly CORS configured. For the p5.js examples uploaded to the editor, the media files are hosted on GitHub and using jsdelivr.
andrewn commentedon Jun 6, 2019
I'm going to close this issue, as the results of the discussion have been incorporated into the Public API proposal (#1076).