11.5 When to Use the Design Verification Methods

The definition section surveyed the types of verification methods possible. The general setup section described the tools available for testing. This section will provide guidance in how to select/craft a verification method given the intention of meeting a requirement or managing risk. Functional testing is to ensure that a component, subsystem, or system meets a performance requirement and does not fail along the way. Functional tests are going to vary with respect to the system under test and the evaluation criteria, for example, the star tracker and quaternion output precision. How does one know which verification style to take? The picture is reiterated for your convenience.

These verification methods ascend in the level of effort and rigor all to mitigate some amount of risk, which isn’t always quantifiable. The line of questioning should begin with: “What level of testing will provide me sufficient confidence that this spacecraft will meet its requirements in space?” For example, given a structure with a finite element analysis factor of safety margin of 10, a simple inspection of the fabricated structure prior to assembly may suffice. The structure is metal and the simulated model characterizes structure behavior well. This component began with a very low probability of failure so the level of effort in verification shall be analogously low. On the other extreme, payloads that are state-of-the-art and custom-built for a mission have a very critical role in a mission and a high-risk posture due to its unprecedented operation. This system would likely need to go through all verification methods, leading up to the culminating testing of hardware and software simultaneously, pushed to the limits of operation to characterize capabilities. Something embedded in this discussion is the idea of heritage, whether a component has been built, functional, or operated in space before.

 A formalized way to think about risk is the possible failure mechanisms or hazards with their associated likelihood. The table above provides some general guidelines as to the qualification test that can identify the potential failure mechanism given a justified suspicion of this hazard. There is a risk that an acceptance can lead to a failure mechanism, shown in the table below. We want our testing to be rigorous yet minimal to prevent undue stress on the spacecraft.

Let’s talk about the Artemis CubeSat Kit’s thought process behind verification methods. For each subsystem, the following objects and conditions are of concern and a justification for a verification method is posed. Historical mission failures should inform you how worried you should be about those aspects failing you. Here’s the original smallsat mission failure survey; please peruse Appendix A: Small Satellite Missions that Partially or Totally Failed.

• Structures:
  • The primary structure is under critical load. The primary structure is aluminum, a well-characterized alloy. The critical load is the compression from the deployer. The point application of the critical load is at the structure’s rails. A finite element simulation result should be sufficient to retire any concerns around the structure’s ability to survive this critical load if the margin of safety is above 1.6. Results show a margin much higher than 1.6.

• • The antenna burn-wire mechanism upon deployment. The burn-wire deployment mechanism is popular and commonly used but the circuit was designed in-house. To verify that the circuit for the burn-wire deployer works, a physical test is implemented: a realistic voltage and current are supplied to the circuit to observe if the mechanism deploys.
  • The integrated structure is under a launch-like vibration profile. The many mechanical joints in the CubeSat are at risk of loosening or disconnecting. Although the quality of integration can be verified through inspection, a much more rigorous test that most launch providers require is a vibration test, which physically simulates realistic vibration profiles.

• • Some reasons for mission failure [NASA]:
    • ISAS & NASDA’s DASH satellite likely did not separate from the main satellite after launch. No contact was established.
    • Deployable solar panels not deployed. Not enough power was generated.
    • Despite many attempts, DOS (DeOrbitSail)’s de-orbit sail could not be deployed.
  • A detailed analysis of structures verification for the ISS modules is written by ESA.

• Power
  • The power distribution unit’s ability to supply sufficient power to the other subsystem components in the worst-case scenario that all components (that do not directly conflict) are on. A downstream result that verifies that all systems are functional is if the onboard computer can read all sensor measurements and command all actuators, and further, that all measurements are sensible and actuators behave sensibly. As this power distribution unit was designed in-house, this design must be rigorously stress-tested.

• • Solar panel’s ability to charge the battery and supply the satellite with enough energy to last the whole mission. The solar panels may function differently than the manufacturer’s reported ideal conditions. To verify the charge rate, a sun simulator should simulate the sun’s irradiance at the CubeSat’s anticipated orbit for a realistic duration of the orbit. The physical test verifies the component’s as-built performance which can then be injected into the power budget and profile. The analysis on the profile with these updated numbers should reveal the solar panels and battery’s capability to supply enough energy to the spacecraft through the entirety of the mission.

• • The battery safety and survivability through vibration tests, as specifically dictated by the JSC 20793 Crewed Space Vehicle Battery Safety Requirements.

• • The integrated structure is under a launch-like vibration profile. The many electrical joints in the CubeSat are at risk of loosening or disconnecting. Although the quality of integration can be verified through inspection, a much more rigorous test that most launch providers require is a vibration test, which physically simulates realistic vibration profiles.

• • A lot of small satellites become inoperable due to power problems. Some reasons may be [NASA]:
    • The battery state of charge is not accurately estimated or the threshold to go into safe mode is too high. The satellite is forced to go into a safe mode to conserve energy when in reality, there is plenty of juice in the satellite but it’ll never leave safe mode.
    • ASUSat-1’s power system prevented solar arrays from charging the batteries.
    • FalconSat-1’s power system was unable to charge the batteries.
    • US Naval Academy and George Washington University’s BRICSat-P had issues with the power system that prevented consistent communication from being established.

• • Communications
    • Previously mentioned: antenna deployment. Mechanisms are always risky so we need to ensure that this mechanism works reliably, otherwise, we will not be able to receive any commands from the ground.

^{Nayif-1 CubeSat Antenna Deployment Test – slow motion. Video by Wouter Weggelaar}

• • Radios establishing a link. A plurality, if not majority, of mission failures, are due to communication failures. The design for the communications system, on the satellite and on-ground, must be verified, at first with a detailed link budget and then with physical testing. The as-built communications system should be physically tested with realistic input power and physical distance between the radio and receiver with attenuation on the ground station side to simulate the additional effects of atmosphere and orbital distance. A monitor at the ground station should be able to pick up a distinct signal from the satellite’s radio and vice versa.

• • Signals are properly encoded and decoded such that the original signal can be interpreted. A step further from just detecting that a signal is received is if that signal can be decoded into useful information. We should know the signal that we’ve sent from the satellite. The signal is then encoded with some modulation and coding scheme, the ground station picks up the said signal, the software decodes it and we should be able to see that original signal. There is a risk that bits are flipped or that the signal is below the noise floor (the signal strength is so small that it is obscured by white noise).

• • Here are some mission failures [NASA]:
    • No signal was received at the ground station.
    • SimpleSat could not be contacted; suspected transmitter failure.
    • DTUSat could not establish two-way contact with the satellite.
    • Russian Academy of Science’s COMPASS-2 spacecraft lost communication after launch due to a stabilization problem. The spacecraft did not respond to ground commands for six months. Although communications with the satellite were restored, a failure with the power system allowed only a very limited amount of data to be transmitted.
    • Tokyo Tech Engineering Satellite’s CUTE-1.7+APD had failures in the communication system after launch that made it impossible to conduct any experiments. A single event latch-up (SEL) is suspected as the cause.
    • The University of Michigan and NRL’s CADRE successfully deployed from ISS, but no signals were received.

• • Command and Data Handling
    • The onboard mass storage memory handles and stores all payload, sensor, and health data. The data estimates for the various components are a sufficient baseline for how to size the mass storage, but a physical test must be conducted to get a more realistic estimate of data generation and storage in real operations. There may be additional bits that must be stored in implementation that are not accounted for in the initial estimate.

• • The data bus and onboard computer need to handle and route all data without losing any bits. Depending on the frequency of sampling and the timing of when these signals must be handled, the onboard computer might experience race conditions in which data is lost or garbled. These issues only come up in implementation so a physical test is necessary.

• • The onboard computer processing needs to be powerful enough to complete processes in a timely fashion such that their products are delivered to programs when they are needed. Given the way, a program is written and the computational burden on the computer, the processing is going to take a range of runtimes that can only be characterized with physical testing. Through testing, the runtime can be shortened and optimized.

• • Here are some mission failures [NASA]:
    • Aalborg University’s AAU-CubeSat-2 operating system malfunctioned. Rebooted 10-14 times daily caused by timing errors on the bus. Flight plan erased and de-tumbling inactivated with every reboot. Some data received showed tumbling above 2 Hz.
    • Delft University of Technology’s Delfi-C3 CDHS design has an inherent flaw that often prevented data transmission on the bus, leading to either insertion of zero’s in the telemetry data, arbitrary switch off of subsystems, a reset of the computer, or even a fallback to a very limited back-up mode.

• Attitude Determination Control and Sensing

Sensor polarities may be integrated into the software or structure incorrectly, like in the Proton rocket. These incorrect mourning configurations or negative signs in software lead to unstable control of the vehicle’s dynamics, which can cause catastrophic failures. For critical events like launch or detumbling, sensor and actuator polarity is critical to test on ground. For less critical maneuvers, this kind of failure can be corrected with updated software.

^{Proton-M rocket catastrophic failure due to wrong sensor polarity. Video by TheMrSuslov}

• • Actuator failure. Some actuators, like reaction wheels and control moment gyroscopes, have moving parts: the rotor and gimbals. These joints can lock up or wear down until dysfunctional. These components rely heavily on space heritage on previous missions because ground testing doesn’t totally represent behavior in space. Ground tests on redundant units can reveal failure modes due to fatigue. For these reasons, spacecraft include redundant actuators.

• • Here are some mission failures [NASA]:
    • DLR’s BIRD had  3 of 4 reaction wheels fail, plus the failure of the gyroscope.
    • Academy of Sciences of the Czech Republic’s MIMOSA never became fully functional due to accelerometer proof mass being able to move freely in only two axes.
    • The Aerospace Corporation’s PSSCT lost spin stabilization caused by eddy currents in the aluminum hull that prevented desired solar cell performance data to be obtained.

• Thermal
  • The Heater’s ability to warm or sinks ability to cool temperature-sensitive components. The amount of surface area and the quality of contact that a sink or heater has with the component we’d like to transfer heat from or to can only be verified physically. Finite element analyses can inform a design, but the implementation must be physically tested. A poor thermal connection risks degraded performance for the element it is trying to cool or warm, whether that’s a battery (which can freeze up if it’s too cold) or a mirror (which can warp if it’s too warm).

• • Radiation-dominated heat transfer of the spacecraft system. Finite element analyses cannot fully characterize all the details and as-built nuances in a satellite system. Further, ground testing in the open air does not characterize radiation-dominated heat transfer. Thermal vacuum chamber tests are the standard to verify a thermal subsystem’s efficacy.

• • Here are some mission failures [NASA]:
    • The University of Applied Science at Aachen, Germany’s COMPASS-1 hard reset put the satellite into emergency mode for several days, causing the heater to fail.
    • Stanford University’s QuakeSat lost its batteries due to what they thought were high battery temperatures (120 degrees Fahrenheit), which may have caused the electrolyte to bake out since the batteries were not sealed beyond the normal factory packaging.
    • Tohoku University of Sendai’s SpriteSat battery charging system allowed the temperature of the battery to reach critical levels