Introduction

The gap between linear and discrete nature of computational systems and the reciprocal nature of creative inquiries poses a significant barrier to employing traditional CAD systems in complex design problems. Unlike these systems, Machine Learning (ML) models can be implemented to contribute to the creative process by enabling the users to explore otherwise hidden latent space. This project is the first stage of a more extensive research, exploring the use of learning-based models in suggestive modeling in creative inquiries. The goal is to develop a Machine Learning (ML) model that is capable of deriving solutions from the latent space based on a problem-specific dataset, given a partial or incomplete design solution generated by the user. Recent works in the area of shape completion offer promising results in reconstructing voxel representations from partial or missing data (Stutz & Geiger, 2020), and we believe a similar approach can be used for the hypothesis generation stage. Such efforts utilize the modular nature of Auto Encoder (AE) networks first to teach latent space representation and reconstruction and then use the trained decoder in another cycle of training to teach latent mapping between partial data and full reconstruction.

By doing so, we hypothesize that these systems can go beyond the role of task executors and take an active role in the design process. Furthermore, by training these learning-based models on problem-specific datasets, we aim to capture design intentions and provide relevant solutions as opposed to generic, cookie-cutter algorithms traditional systems incorporate. We believe that changing our perspective on CAD systems from automation and optimization to idea generation and collaboration can facilitate new human-machine interaction scenarios in creative domains.

Background

Historically, the field of design has been influenced by new techniques and technologies for design representation and fabrication. Arguably, one of the most influential of such technologies has been CAD since its conceptualization in the early 60s. Even though early visions for CAD were to create a collaborative partner in creative exploration, the course of future research shifted from developing a collaborative partner to “perfect slaves’’ poised to liberate designers from arduous tasks, This change in perspective crystalised the instrumentalist perspective surrounding the computer aids to design discourse.

Current literature that investigates 3D ML models offer promising results in low-dimensional representation and generation of 3D objects by appropriating well established models. Some of the most popular models used in conjunction with 3D data are Variational Autoencoders (VAE) and Generative Adversarial Networks (GAN). These models are used in a wide range of problems from scene segmentation to object classification in different applications such as autonomous vehicles and robotics.

One area that research in geometric learning focuses on is how the geometry is represented within these models. Currently, there are three mainstream representation methods for handling 3D data representation in learning-based models; (1) voxels, (2) point clouds and (3) meshes. The underlying differences between the three methods lie in the application, operation, and representation. Voxel-based approaches are prevalent in ML applications as they are inherently very similar to multidimensional inputs such as images. However, since the representation relies on a grid-based data structure, this approach is limited in its ability to scale. Point clouds, on the other hand, define each point as a set of cartesian x, y, z coordinates. It is one of the most popular ways of representing 3D data as it offers more flexibility with scaling and can easily be worked into mesh geometries using computer graphics algorithms. However, since point clouds are often unorganized in terms of neighborhood information, this representation model needs to be approached carefully, especially in conjunction with ML models. Lastly, the mesh representation stores data in polygons and vertices that make up the faces of a volume. This eliminates the problem of unorganized data structure seen in point clouds. However, since this approach incorporates the most detailed data, the operations can quickly become computationally heavy.

System Architecture

In machine learning, dimensionality reduction is the process of reducing the number of features that describe some data. This reduction is done either by selection (only some existing features are conserved) or by extraction (a reduced number of new features are created based on the old features) and can be useful in many situations that require low dimensional data. Dimensionality reduction can be interpreted as data compression where the encoder compress the data (from the initial space to the encoded space, also called latent space) whereas the decoder decompress them. The general idea of autoencoders is pretty simple and consists in setting an encoder and a decoder as neural networks and to learn the best encoding-decoding scheme using an iterative optimisation process.

Figure 1. Implemented AE architecture.

The primary approach employed in this research comes from the prior work done by Achlioptas et al. in their paper titled “Learning Representations and Generative Models for 3D Point Clouds” (arXiv | GitHub), where they explore the data-driven design capabilities of generative models introduced in the previous section, including a VAE architecture. In this study, the authors were able to reconstruct and generalize the 3D dataset to interpolate and generate 3D shapes by training a deep Auto-Encoder using point cloud data inputs. This architecture learns to encode the input 2048 points (2048x3 coordinates) into a latent vector of size 128 using 1D convolution filters activated by Rectified Linear Units (ReLU) function and consequent max-pool layer. Using fully connected network structure, the decoder network reconstructs the original point cloud from the given latent space vector (Figure 1). In order to negate any negative effects of point cloud permutations, the authors adopted Earth Mover’s Distance (EMD) and Chamfer distance metrics, which are commonly used in 3D learning models due to their permutation-agnostic approach. While our approach follows a similar path established by Achlioptas et al., we extend the application of this method in latent design space by taking a narrow design space and extrapolating the dataset. As a proof of concept, we focus on a simple vase topology that is addressed by different design approaches and enable the latent design space that places emphasis on subjective design over objective design.

Dataset

Figure 2. Parametric model developed for synthetic data generation.

This model was initially tested using the point cloud dataset provided by the authors, which consists of uniformly sampled point clouds from a subset of the ShapeNet database. This step was necessary to validate that both the operating system and driver dependencies were met. Following the successful training and exploration of the model with the default dataset, we have shifted our focus to developing parametric scripts on Grasshopper for Rhinoceros 3D that is capable of generating the design space for a vase topology (Figure 2). This parametric model is composed of seven distinct parameters that generate a NURBS curve to create a surface of revolution. This surface was later sampled uniformly to generate point cloud data for training the AE network. In order to broadly capture the design space using these seven design parameters, we have generated 3^7=2,187 synthetic data points, sampling each parameter range three times. Since the range of the parameters are tied to each other, this approach resulted in unique designs that vary in height, maximum and minimum radius, global curvature and top and bottom radii. Figure 3 showcases some of the datapoints generated by this approach.

Figure 3. Series of datapoints generated using the parametric model for single class dataset.

In addition to broadly capturing a design space, we have also developed three additional scripts that apply different local geometric features to these surfaces. These geometric features are corrugation that creates a ridge and groove pattern, asymmetric that deforms the geometry by introducing random attractor points that pull and push the surface, and low polygon that reconstructs the original geometry with planar facets. Each design approach is randomly generated 2000 data points, totaling 8000 distinct geometries together with the simple surface generation (Figure 4).

Figure 4. Series of datapoints generated using the parametric model for multi class dataset.

Results

With these two datasets, our goal is to understand the capabilities of the AE model in learning both local and global geometric features. The single class dataset that captures the entire space by sampling all the possible variables is used to explore whether the AE model used in this study is capable of capturing global geometric features of the design space. Within this experiment, we used the reconstruction and interpolation accuracy as our metric for evaluating the success of the AE model. We calculate the reconstruction deviation, using the initial surface as the ground truth. Similarly, we can investigate how well each parameter is captured by the model using the parametric model interpolation as the ground truth.

Figure 5. Interpolation accuracy between single class dataset points.

Figure 5 above illustrates three interpolation operations between six distinct data points. From these results, we can observe that the AE network yields a very accurate reconstruction of the input point cloud. For these data points, the maximum deviation from the ground truth was less than 0.001 units. However, during the interpolation between known data points, we can see that the AE network does not necessarily follow the geometric rules defined in the parametric model. Although this might be considered an error based on numeric metrics such as the one we use, we believe that this behavior can be beneficial in creative inquiries. It opens up the design space further based on the importance of different learned geometric features.

Figure 6. Interpolation between multi class dataset points. From top to bottom: “simple” to “corrugated”, “simple” to “asymmetric”, and “simple” to “lowPoly”.

The reconstruction accuracy for the multi class dataset, on the other hand, provides interesting insights into the capabilities of the AE network. While the model can reconstruct the “simple” and “corrugated” classes near perfectly with a maximum deviation of 0.0005 units, as more random features are introduced, such as the asymmetric deformation and low polygon representation, the reconstruction accuracy considerably decreases. The examples of the “asymmetric” class reconstruction in Figure 6 yielded a maximum deviation of 0.002 units. We can also see that the most extreme deviations are clustered around where the geometry is deformed the most. Similarly, the “lowPoly” class suffered from reconstruction accuracy, as it failed to capture the facets of the low polygon representation and instead smoothed out the reconstruction into a more continuous point cloud. We can see that the reconstruction errors are clustered around the facet centroids, which signifies that the AE model failed to grasp minute local features.

Figure 7. Reconstruction accuracy for multi class dataset.

The t-SNE plot generated using latent vectors of the multi class dataset shown in Figure 8 provides an insight on how the AE model learned the dataset. From this plot, we deduce that the AE model primarily learns global geometric features which results in heavily overlapped latent space vectors.

Figure 8. t-SNE plot for multi class dataset/