Effectiveness of Using Remote Laboratory in Promoting Simulation and Verification Tools

AI summary

Overview

Research area: Engineering education and remote/online laboratories — specifically FPGA-based digital design instruction, evaluated through surveys of students and industry professionals. The paper is posted under arXiv's cs.CY (Computers and Society) listing, presented here under an AI Safety & Ethics category, and its own keywords are "Remote Laboratory, Simulation and Verification tools, Evaluation of Online Labs."

Technical level: Intermediate. The technical apparatus is a virtual breadboard interface to remote FPGA hardware (DE1-SoC, GPIO/JP1, LEDs, switches, HEX display), but the core of the paper is a survey-based evaluation, so no deep FPGA design expertise is required to follow the argument.

One-sentence scope: The paper evaluates a virtual breadboard that gives students remote access to FPGA hardware, and pairs a student survey with a survey of 35 industry professionals to argue that remote labs strengthen the simulation and verification skills employers want.

What This Paper Is About

Teaching FPGA courses has traditionally required students to be physically present in an on-campus lab with workbenches, PCs, and auxiliary tools. The Covid-19 pandemic made that impossible, and shipping physical DE1-SoC lab kits to students was logistically complex. The paper describes a virtual breadboard that lets students assemble and wire circuits online, compile their designs, and run them on real FPGA hardware at the University of Washington, then evaluates whether this approach — and the simulation-heavy workflow it forces — matches what industry expects from graduates.

Key Contributions

1. A virtual breadboard interface for remote FPGA labs. Students place and wire components (three switches, two LEDs) on an on-screen breadboard, connect them through a GPIO header linked to the JP1 of the DE1-SoC board, and preserve all connections automatically to mimic real breadboard behavior. A pinout diagram for GPIO 0 (JP1) shows the relationship between pin names and circuit construction.
2. A remote lab deployment inside and outside the course. The remote FPGA setup lives in the Remote Hub Lab (RHLab) and is reachable through a global network called LabsLand. The virtual breadboard was used by students in the junior-level Design of Digital Circuits course at UW in Autumn 2020, in addition to the newly designed interface, because physical kits could not be shipped.
3. An industry-facing survey instrument. A new survey was built to capture how professionals who use modeling and simulation in their jobs view simulation and verification tools, mixing Likert items on a 1–5 scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree) with short-answer questions.
4. A cross-comparison of student and industry perspectives. By setting the student survey (reported in Li et al., 2022) against the industry survey, the authors conclude that remote labs push students to spend more time on simulation tools than traditional offline design does, improving their simulation skills — a finding they say aligns with Hussein and Wilson (2021).

Main Findings

• The virtual breadboard replaced the physical breadboard. According to the student survey reported in Li et al., the virtual breadboard served its goal as a replacement for the physical breadboard, and deliberate design choices such as the placement of the pinout diagram helped students understand course topics. Students found it simple to use and became adept quickly, saving time for other activities, though the survey also showed room for improvement in the breadboard interface.
• Industry respondents are overwhelmingly tool users. Of the 35 industry professionals surveyed, only two had not used any simulation or verification tools in their job duties.
• Respondent mix. Forty-nine percent of participants are engineers working in related areas; the remaining 51% are managers.
• Simulation is how hardware verification gets done. Respondents rated "verification of hardware design is typically done using simulation tools prior to deploying a functional solution on real hardware such as FPGAs" at a mean of 4.06 (min 1, max 5, SD 1.03, variance 1.06), and "entry level hires are expected to have solid skills in using simulation and verification tools" at a mean of 4.06 (min 2, max 5, SD 1.00, variance 1.00).
• Simulation is seen as an efficiency gain. "Using simulation and verification tools makes the design phase more efficient" was the highest-rated item, with a mean of 4.49 (min 3, max 5, SD 0.66, variance 0.43).
• Teamwork, data handling, connection, consistency, and assessment all rated positively. Mean scores were 4.11 for working effectively in a team (min 3, max 5, SD 0.87, variance 0.75); 4.37 for facilitating data handling, with AWS and 1010data named as examples (min 3, max 5, SD 0.69, variance 0.48); 4.21 for connecting users across a large variety of teams to carry out experiments (min 3, max 5, SD 0.69, variance 0.47); 4.29 for obtaining consistent experiences (min 2, max 5, SD 0.80, variance 0.64); and 4.09 for making feedback and assessment readily available (min 2, max 5, SD 0.83, variance 0.69).
• Responses were tightly clustered near the top of the scale. The authors note the minimum column is effectively an outlier and that the variance is slight, with average scores close to each question's maximum value.
• Time on simulation tools differs between modalities. Comparing the two surveys, the authors state that the remote lab forces students to spend more time on simulation tools than traditional offline design, so students considerably improved their simulation skills.
• Work split between tools and hardware. Asked how much they work with verification/simulation tools versus real hardware, responses ranged from 0 to 100%, with a mean of 0.54 (SD 0.33, variance 0.11).
• Examples of online experimentation environments named by respondents include EDA playground, the web inspector in the Chrome browser, Browserstack, Citrix VDI, MATLAB, AWS, Google Cloud, TWCC, the Xcode iPhone simulator, Labview, ORAL__RT, typhoons, Bento Notebook (developed by META), and internal tools giving access to FPGA, new PCB designs, and ASIC boards and servers.
• Not reported: This paper does not report the numeric results of the student survey; the authors direct readers to Li et al. (2022) for those, and they acknowledge that the industry survey data is limited in scope.

Methodology in Plain English

The researchers built an online interface that looks and behaves like a physical breadboard. Students drag components onto it, wire them together, and those connections stay in place, just as they would on a real board. The board talks to a GPIO header attached to the JP1 of a DE1-SoC board, and students write SystemVerilog designs in a web-based code editor. When they compile, they can flip the switches and watch the LEDs and the HEX display on a live video stream from an actual FPGA housed at the UW campus. A tutorial walks them through the workflow.

They then ran two evaluations. The first, previously published in Li et al. (2022), surveyed students in a junior-level Design of Digital Circuits course at UW in Autumn 2020 about their experience with the remote laboratory. The second, new in this paper, surveyed industry professionals about simulation and verification tools. That survey went to three kinds of partners: companies that develop such tools (for example Intel and Cadence Design Systems), companies that use them (for example Qualcomm and Samsung), and other engineering-related companies (for example Google, META, Amazon, and Apple). Thirty-five professionals completed it, answering Likert items from 1 to 5 plus short-answer questions about their current or previous jobs. The authors then read the two sets of results side by side.

Why This Matters

Impact on research. The paper argues that remote laboratories are not a pandemic stopgap but a durable educational platform. It contributes a reusable evaluation instrument for industry perspectives on simulation and verification skills, and it links remote lab design choices to workforce readiness, echoing Hussein and Wilson (2021).

Real-world applications:

• Remote delivery of FPGA and digital design courses when students cannot be on campus or when lab kits cannot be shipped.
• Giving online engineering students anywhere-and-anytime access to advanced electronic and communication equipment, in the spirit of the Remote Laboratory at the Open University of Catalonia (RLAB-UOC) discussed in the paper.
• Simulated-plus-real hybrid GPIO labs, where students wire circuits freely rather than being limited to the pre-wired switches, buttons, and LEDs typical of existing boards.
• Shared access to hardware through a global network such as LabsLand, so one physical FPGA at a hub site can serve many students.

Industry relevance. The survey respondents — engineers and managers from tool vendors, tool users, and other engineering companies — rate simulation and verification skills as expected of entry-level hires and as the standard way hardware design is verified before deployment. The authors conclude that new employees should be well trained in these tools at school, and that remote labs are well aligned with industry practice because they provide consistent results, easy access to analyzing and interpreting data, and opportunities to develop teamwork abilities.

Future Directions

• Expand the virtual breadboard so students can investigate logic gates and use an increased number of general-purpose inputs/outputs (GPIOs).
• Develop a more advanced virtual breadboard that incorporates microcontrollers or FPGAs.
• Conduct further interviews with industry experts, since the current industry survey data is acknowledged to be limited in scope, to reach more definitive conclusions about the significance of simulation and verification tools in industry.
• Evaluate additional lab assignments to produce a comprehensive statistical analysis of the benefits of remote laboratories.

Target Audience

Engineering educators and lab administrators deciding how to teach FPGA and digital design courses online or in hybrid form; researchers in engineering education working on remote and virtual laboratories; curriculum designers who need evidence linking classroom tooling to employer expectations; and industry partners or hiring managers interested in how graduates are trained in simulation and verification tools. Students entering such courses will also benefit from understanding what the remote workflow demands of them.

Authors’ abstract

The transition to remote learning during the pandemic has necessitated the development of new methods for conducting hands-on experiments. One significant challenge in this transition has been providing students with reliable and sustainable access to necessary hardware components, particularly for courses that require substantial equipment. Additionally, industry partners have high expectations for students to be proficient in simulation and verification tools. To address these challenges, we implemented a virtual breadboard feature that allows students to remotely access Field Programmable Gate Array (FPGA) hardware and complete lab assignments. Our evaluation of this approach, which included surveys of students and industry partners, revealed that it effectively transformed a traditionally in-person lab assignment into an online modality. Furthermore, this paper presents the perspective of industry professionals on verification and simulation tools as a highly desirable skill in the industry, a skill that remote labs tend to emphasize which makes remote labs a viable educational solution that can continue to be utilized even after the pandemic.