res·o·nance: entry #2

an update, &

my explorations with self and mutual capacitance

As mentioned in my last post, I wanted to make a separate entry regarding my learnings and experiments with touch sensors, specifically self and mutual capacitance-based setups. Well, here it is! But first, I wanted to give a bit of an update on the progress of the project as a whole, as well as an exciting announcement.

first: the update!

On March 20th, I woke up to an exciting email - I had been accepted into the Blackbird Protostars Season 9 cohort! When I wrote the first entry post for this project, I was in the middle of applying for the program, knowing that the grant would go a long way towards developing this to the scale I envisioned it could be. I'm so grateful to Blackbird Foundation for recognising the potential of this project and for having me in this season! So far, we've had the weekly kickoff, which allowed me to meet some incredible young people working on amazing projects spanning hardware, game development, storytelling and more. The program will officially end with a showcase on May 13th, at which point I hope to have developed at least one full-scale installation, with other ideas in the works as well.

On that note, I have in fact come up with some more concepts and further developed my existing ones:

I'm not feeling too strongly about a flat-laying kinetic sculpture - it's not really the immersive experience I envision it could be - and so I've re-imagined it as a hanging sculpture through strings hanging from the ceiling.
1. This may not be feasible depending on location, so I've put this idea somewhat in my back pocket for now!
For the light installation previously mentioned, I fear that creating a large-scale mutual capacitance touch sensor won't work, so as a backup plan I've envisioned creating a row of different, smaller light installations that each have a different sensor behind them.
1. The idea is that participants can explore how different sensors work and see the feedback from their interactions firsthand - sort of like a museum but for sensors.
2. I'd still love to do a large-scale light installation, but perhaps with a more reliable sensor driving the interactions. This will depend on prototype testing (more on that below!)
A new concept I've come up with is a soundscape that changes slightly depending on who's in the room, eg. through facial or fingerprint recognition.
1. (Unless the sound is recorded) one person would not hear the same soundscape that someone else hears when they're each alone in the room, but together they hear a new unique sound, representing their shared experiences together even through having unique identities.
2. I want the sounds to be organically created in some way, such as through actuators semi-randomly driving percussive elements (changing frequency or some other factor depending on who's in the room). Essentially hardware-driven, rather than sounds coming out of a speaker.
Another new concept is more of a commentary, specifically on AI art and its impact on artists and photographers. This would involve having a photograph I've taken up on a screen, and an AI image generator next to it. The challenge would be to recreate the photograph through giving the AI model prompts, but every time a prompt is entered, a pixel is removed from the original photograph.
1. The message I want to send is two-fold: 1. that AI art cannot recreate the same hard work, emotional intent, skill, and relationships that real artists put behind their work, and 2. that we are all complicit in this AI takeover of art through using machine learning models in our everyday lives. Regulations of AI and how they're being used in artistic and creative spaces must be put in place before they're so freely used by the public.
2. This was inspired by the many artists and photographers I see online sharing how they've witnessed people recreating their works using AI, or seeing comments on their work of people calling it AI, and the damage this does to their reputation and self-worth as artists. Personally, I would feel highly offended by someone recreating my photograph using AI without my permission, because it in no way replicates the time that I've put into creating my backdrops or choosing locations, setting up lighting, working with models and friends, and carefully editing and curating my works. I enjoy all these aspects of photography and do not wish to have any of them replaced by a machine.
3. Given this is a bit of a bleak message, I had the idea of adding a photoshoot setup where people could create their own images, and each time a photo is taken, a pixel would be added back to the original photograph. The idea here is to demonstrate how fostering human creativity can intervene in this AI takeover :)
One final concept I have in mind isn't quite fleshed out yet, but it's based on the art of making zines and the driving factors behind them that are community and storytelling. This one needs some more thought first, so I'll be returning to it as it develops in my mind!

Once these ideas are more fleshed out, I plan to create a mockup (perhaps something similar to an architectural design mockup) of the entire exhibition to better illustrate my ideas and how they will flow together as an overall experience.

Long rant about my ideas is now over! Onto the next thing...

self and mutual capacitance: research

I've done some pretty extensive research on the basics of capacitive touch sensors, specifically those used for multi-touch applications such as touchscreens, and how this can be applied on a practical, DIY level. In the spirit of free, open-source building and learning, I thought I'd share my findings here in detail!

From my research, I learned that there are two main types of capacitive touch sensing: 1. self-capacitance, and 2. mutual capacitance. Both concepts rely on the ability for a human finger to change the capacitance of objects, depending on its proximity to that object. However, these two types of capacitive touch sensing work slightly differently:

Self-capacitance: The touch sensor forms a capacitor to ground, and a circuit measures changes in its capacitance value when a finger is placed on it (i.e. the finger acts as a parallel capacitor to ground, thus adding capacitance to the sensor).

More details if you're curious:

Approaches are often implemented to accurately control and measure the charge applied.
1. For example, a charge time measurement unit (CTMU) applies a constant charge to the sensor for a fixed period. Once this period ends, the voltage on the sensor is measured using ADC, before the sensor is then discharged and the process repeated.
Self-capacitance is typically used when a small number of buttons are required, or for proximity detection.
1. Sensors can be grouped together to form rotary interfaces or sliders.
2. Interleaving the sensors can also provide smoother results!
3. Sizes of sensors should ideally match a human fingerprint (8-20mm in diameter).
Sensors are usually placed behind a plastic cover, but the thicker the material, the lower the sensor sensitivity.
The measured signal will often be affected by interference.
1. Interference can be addressed through filtering, through adding ground rings around the sensor, or a hatched round behind the sensor.
2. However, the sensitivity shouldn't be reduced too much!
  measured signal will be affected by interference - some filtering necessary
Parasitic capacitances to ground often have the effect of desensitizing the sensor to a user's touch: due to the additional capacitance, the percentage change in capacitance when a human finger is present will be much smaller than if the parasitic capacitance wasn't there.
Self-capacitance electrodes project electric field lines 360 degrees outwards 360 and can be interacted with on both sides unless ground shielding is utilised.

Mutual capacitance: Works similarly to self-capacitance, but instead of the second "plate" of the capacitor being ground, we define the second plate: essentially there are two electrodes (TX and RX typically), which the finger "steals" charge from, causing a drop in capacitance across the sensor.

More details if you're curious again:

This method, often used in smartphones, allows for the creation of a grid of capacitors across flat surfaces. This is often referred to as projected capacitive touch, or PCAP.
For example, one implementation involves charging X rows of the sensor area sequentially, then evaluating the capacitances using Y rows, with each cycle occurring tens of times per second.
1. Ideally, both the X and Y row connections are on the same side.
In smartphones, this pattern is applied to the glass cover above display using Indium-Tin Oxide (ITO), a transparent material with low resistance.
Dedicated chipsets are often used, eg. MaxTouch.
At output, the user interaction is reduced into X-Y coordinates for each finger, allowing for gestures to be mapped out and used (eg. pinch, swipe, rotate).
A note on parasitic capacitances:
1. Parasitic mutual capacitance: Anywhere that a TX trace comes near an RX trace, the two will have a parasitic mutual capacitance - similar to parasitic ground capacitance in self-capacitance.
2. Parasitics to ground (CpTx, CpRx): In a mutual capacitance network, the TX and RX electrodes will still have capacitance to the circuit ground and earth ground, but this doesn’t affect the measurement in this case. It does still affect the circuit's ability to drive the electrodes at high speeds since we are charging and discharging that capacitance.
Benefits over self-capacitance:
1. Because mutual capacitance electrodes are defined by both "plates", the electric field that a user can interact with is tightly defined between these two conductors (unlike in self-capacitance where the field of interaction projects outwards in all directions). This allows for keys to be closely grouped together without the possibility of cross-coupling or ghosting effects - useful if a user isn't perfectly centred over one key - and it's easier to route guard channels and proximity sensors between keys.
2. In other words, a matrix of sensors can be implemented using mutual capacitance, with columns as TX electrodes and rows as RX electrodes. Essentially, each row/column intersection point becomes a unique TX/RX combination, and a unique mutual capacitance, allowing for the implementation of multi-touch across the grid.
An overlay material is often required for mutual capacitance touch panels, providing an area for the electric field between the RX and TX electrodes to propagate for user interaction. Without this, the RX and TX connections would make contact and short, adding significant parasitic capacitance to ground.
1. The overlay thus projects out the mutual electric field.
2. Electrodes must be shaped according to the overlay material to optimise sensitivity: if the electrodes are too close together with a thick overlay, there will be little field propagation to the top of the overlay.
The change in capacitance due to a touch on a mutual capacitance electrode is usually smaller than that of a self-capacitance electrode.
1. As a result, measurements tend to be noisier for mutual capacitance, making it difficult to construct large sliders and wheels unless many electrodes are used.
2. Mutual capacitance electrodes are best for button matrices and more advanced sensors with small node sizes but high electrode density.

To detect the change in capacitance on a practical level, a counter/timer can be used:

Either by counting the number of pulses in a specified time period and setting a count threshold,
Or measure the variation in ON-pulse widths.

TL;DR: mutual capacitance can be used to create multi-touch grids, but self-capacitance is typically easier to implement. I wanted to explore both and see exactly how difficult it would be to implement each...

self and mutual capacitance: experimentation

From research, I learned (and confirmed through trying it!) that a simple self-capacitance touch sensor could be implemented using the following process:

Place a large resistor across two pins on a microcontroller (from memory, I think I used 10k ohms) and connect one of the pins to a piece of foil or other conductive material (see the below circuit diagram).
In this case, pin 2 is the "send" pin which charges the sensor, and pin 12 is the "receive" pin which reads the voltage with respect to ground.
Using code provided by DigiKey, and a commonly available library called CapacitiveSensor, pulses are sent out to measure how long it takes to receive an answer, essentially charging and discharging the capacitor. The result is normalised, thus returning an arbitrary number that describes the sensed capacitance.
1. This value then correlates to how close a user comes to the metal object being used as the sensor.

Using a piece of foil, I found that I could actually create a really big sensor surface with this method, which I thought was interesting! But the value returned by the library often fluctuated (which I guessed was due to the varying capacitance of the foil and my hand, perhaps affected by the carpet I was sitting on).

Moving on to mutual capacitance, I found a resource that goes into detail on how to implement a multi-touch capacitive sensor by the Human Computer Interaction Lab at Saarland University. I decided to try creating a single electrode mutual capacitive sensor using electric paint based on the implementation methods discussed in their Multi-Touch Kit manual. It generally worked pretty well (image shown below), and the sensor even works touching it on either side of the paper (no matter where the paint is)! This makes it easier to separate the TX and RX electrodes, so I'm excited to expand this further.

However, there's some improvements I want to try implementing in order to expand this to more electrodes:

Creating a stencil (likely lasercut) to paint a more accurate surface, as it was very difficult to paint the electrodes by hand,
Using copper tape to secure the jumper wires to the electrodes, rather than holding them to the paper.

And, as you might have guessed, those are my next steps for implementing this sensor for the light installation! I want to create a proof-of-concept 4-electrode sensor (essentially requiring two rows and two columns of 3 electrodes each), and have each electrode control a "zone" of lights on a small wire tree to represent the final full-scale design. I've created a rough stencil shape in Adobe Illustrator using 1cm-sized diamond electrodes (image attached below) but have yet to lasercut it - so stay tuned for updates as I continue developing this concept into a full-scale installation!

Sources:

Till next time!

~ Sophia