Create a 3D Editor in SwiftUI and SceneKit | by Mark Lucking | October, 2022

Using Scene View running within SwiftUI Interface

a 3D editor

I was working on an article that relied on a type of 3D editor to set up a simulation, and – well – more than halfway through, it became clear that the simple editor I had made was just this Wasn’t going to cut it. So I decided to document a reconstruction of my journey, said the Ordinary editor. Join me in this story and learn what I made, how I made it, why it didn’t work, and how I fixed it.

Bon – So here’s the rub. The goal was to build a 3D editor that I could use to create images like this, with the challenge of placing cubes. Although I didn’t know it at the time, I’d go through and fix seven problems.

a simple 3D shape

I started with a basic SceneKit implementation that I copied from this article. I needed to use representable because I needed/wanted touch. Anything that moves when you touch the screen, you get a new cube, a cube that you can drag around the display. A drawing that you can rotate to get the cubes into place. As I did in the past, I used the Membership model from the Combine Framework to connect the view to the SwiftUI interface.

It had a good start, at least with a bang. The view was spot on, as were the touch controls. That was the first challenge. Sure I could detect where in my scene a touch event had occurred, but it was in the wrong coordinate location; I needed a routine to translate between the two, which was easier said than done. I found the solution on SO, this code.

Convert screen coordinates to SceneKit coordinates

But I want some structure there, some order. Letting the artist place the cubes anywhere quickly turned into a mess. So implemented this code to try to get things in line.

code to try to put things in more order

But with the structure came a new problem; It was very easy to stack the cubes on top of each other. I need to implement some code to make the cubes avoid each other. Initially, I placed them in different Z planes. But as I moved the camera, the planes changed location, so the cube placement started to look random; it was a mess. I solved it by using worldPosition And another gem from SO.

Code to detect colliding cubes

And this code to find out which side of the cube was already there, I needed to insert new one.

To find out the code that was in front.

I could now place the cubes and rotate the shape I created, but I wanted to be able to move the cubes if they looked like they were in the wrong place – so I introduced a pan gesture to do this.

But there were two problems that surfaced almost immediately with this code. Firstly with the pan gesture, I could no longer rotate the drawing; Even worse, I suspected there was too much interference from SwiftUI gestures; Sometimes it will work, sometimes not.

So I looked at it again, implemented it directly in SwiftUI using drag, back to my sceneView class with membership.

Of course, everyone makes mistakes, and I needed the means to remove a cube if I accidentally added a cube. I did this with this code using a long press gesture.

I was almost there, although I needed a means to change placement that was a little less freehand than drag.

So I decided to try double tap to do so; with this code.

The solution here is almost the same as the previous one, so I leave the code for now. Problem; I wanted to be able to zoom in and out. A requirement that I solved using two buttons that relied on worldPosition To know which direction they need to move the camera.

All this brings me to the end of this short letter. I hope you found some interesting code bytes here; Getting everything working together was a challenge.

Leave a Comment