Intro by Victor
- don’t be fooled by predefined industry solutions, understanding universal programming tools is much more powerful
- Don’t let the tools shape your practice, master the tools that fit in your practice
- Languages like JavaScript, Python etc. are preferred over specific tools like Processing, NodeRED or Blender
- If you understand the programming part, your tools are universal
- Ultimately it’s all about data, maybe more important than UX
→ Separating Front- and Backend is not always the best idea
- Even *machine learning* are just a way of processing data
Highlighted Tools
## Processing
- uses Java, computes locally
- It’s built for the electronic arts, new media art, and visual design communities
## P5.js
- evolved out of Processing to serve online data inputs
💡 The evolution of tools went from local to the network, but now as privacy and ecology are becoming important factors, ‘local first’ becomes more relevant again. So ask yourself in your project: “Is online connectivity and cloud computing really necessary” Is the project reliant on all these steps, if it’s ‘just about turning on an LED’.
## Blender
- Has python scripting included, so anything you can do in the software is also accessible through python
- That means interfacing with microcontrollers becomes very easy
## MIT App Inventor
- For Android
- if you want OS-compatible apps, built a web app
## A-Frame
## Node Red
Serial Communication
- The way you format data is dependent on how complex your data is
→ for a simple array of numbers, a JSON file is not needed
💡 Send the least data as possible (contrary to the industry paradigm, which is so send as much data as possible). This also tends to preserve the privacy of the users more easily</aside>
Writing an Application in Python
To prcactice embedded programming and creating interfaces, we opted for the most direct and straight-forward user experience. In order to control the web-based storytelling generator built during our third challenge week, we chose command line inputs as the most intuitive solution. The python script listens for inputs via the command line, specifying the number of jumps as well as the starting URL (seen below). Through these inputs the user can specify where the crawler should start its journey and how long the journey should take.
More on the program architecture and the functionalities in the related blog post or on Github.
The command line asking for starting URL and the number of jumps, in this case 'wikipedia' and '5'.
API response from GPT3 can be seen, telling a compelling story about a dolphins.
Further Links
Interface and Application Programming
languages
hello world
C, GCC, C++, GDB, DDD
.NET, C#, Mono
Go, Rust
Flutter, Dart
Kotlin, Swift
Haskell, Scala, Julia
Bash, Tcl
Perl, Ruby
APL
Python, tutorial, environment, PyCharm
Processing, Wiring, Arduino, p5.js
Java, OpenJDK, IcedTea, Tomcat
JavaScript
tutorial, Node.js, V8, npm, asm.js, WebAssembly, CoffeeScript
LabVIEW, Simulink, Max, Pd, Scratch, App Inventor, Grasshopper, Firefly
Node-RED, mods
device interfaces
RS/EIA/TIA 232/422/485, pySerial, serialport, Web Serial
I2C, CircuitPython, smbus2, i2c-bus
FTDI, libFTDI, python-ftdi, ftdi
Firmata
USB, PyUSB, usb
IrDA, python-irda
GPIB, VISA, PyVISA
MQTT, XMPP, IFTTT, UPnP, Wemo, RainMaker
socket, dgram, net, ws
hello.mag.45.html
hello.mag.45.js
video
mods
data interfaces
flat files
Calc, Sheets
Pyspread, Pandas
MySQL, MongoDB
user interfaces
ANSI escape codes, ncurses
Tk, Tkinter
hello.load.45.py
wxWidgets, wxPython
panel_png_path.py
Qt, PyQt
GTK, PyGTK
Clutter, PyClutter
Kivy
forms
jQuery, dat.GUI, Bootstrap, Flat UI, Material
Backbone, Require, Angular, Handlebars, Ember, Webpack
Meteor, Babel, React, React Native, Cordova, Ionic, Electron, Blynk, Firebase
graphics
X Windows
xline.c, video, ximage.c, video
$ gcc xline.c -o xline -lm -lX11
AWT, JFC, Swing
JavaLine.java, JavaLine.html, video
JavaImage.java, JavaImage.html, video
Canvas
canvasline.html, canvasimage.html
SVG
svgline.html
WebGL
webglline.html
Three.js
threejsline.html, threejsimage.html, threejssurf.html
OpenGL, GLUT, GLEW, GLFW, Dear ImGui, PyOpenGL
glimage.c, video, glsurf.c
,
video
$ gcc glsurf.c -o glsurf -lm -lGL -lGLU -lglut
glimage.py, glsurf.py
RenderMan, cgkit
VTK, pyvtk, Mayavi
ParaView
OpenVDB
Unity, Unreal, Godot, Blender Physics (video)
audio, video
SDL, Pygame, PySDL2
openFrameworks, ofpython
TouchDesigner
SuperCollider
HTML5
Web Audio
audioline.html
WebRTC
video.html
video
FauxmoESP
VR/AR/MR/XR
WebXR
THREE.VRController
ARCore
Forge
three.ar.js
three.xr.js
AR.js
A-Frame
math, machine learning
Netlib, BLAS, LINPACK, LAPACK
MATLAB, Octave
NumPy, SciPy
matplotlib, Seaborn
line.py (video), lines.py (video), image.py, images.py (video), surface.py (video)
Anaconda, IPython, Jupyter, Colab
plot.ipynb
plot.html
R, RPy, ggplot2
Mathematica, SymPy, Sage, Scilab
Math.js, Science.js, numbers.js
Plotly
Python
JavaScript
plotline.html
D3, jqPlot, Highcharts, Chart.js, mpld3, Bokeh
Theano, PyTorch, Keras, TensorFlow, TensorFlow.js
,
TensorFlow Lite, ESP-DL, Edge Impulse
signal processing, mathematical modeling
performance
pi.py, numpi.py
Cython
Numba
numbapi.py
JAX
pi.html
JIT, typed arrays, web workers, file readers
pi.c
OpenMP
mppi.c
MPICH, Open MPI, MVAPICH
mpipi2.c
mpi4py
mpipi2.py
cudapi.cu
CUDA, PyCUDA
OpenCL, PyOpenCL
WebGPU, GPUMath.js
deploy
REST, PWA
Amazon AWS, EC2, Lambda, ParallelCluster, SageMaker, Honeycode, remote desktop
Google Cloud, Apps Script
Microsoft Azure, Power Apps
DigitalOcean
Linode
Heroku
Docker, Kubernetes, Auto DevOps
security
attack surfaces
transparency vs obscurity
dependencies, patching
encryption
assignment
individual assignment:
write an application that interfaces a user with an
input &/or output device that you made
group assignment:
compare as many tool options as possible