Abstract* Background Vocational guidance is a critical intervention in educational transitions, yet traditional approaches rely on static assessments with limited personalization. Artificial intelligence (AI)-based systems offer the potential to integrate psychometric profiles, academic data, and contextual variables to generate individualized recommendations. This protocol describes the evaluation of GoaleTest, an AI-driven vocational guidance system that matches students’ psychological profiles based on Holland’s RIASEC model with undergraduate program options using a proprietary nonlinear vector alignment algorithm that incorporates signal amplification, antisignal penalization, and nonlinear affinity mapping. Methods A quasi-experimental longitudinal design will be employed. Participants will be 200–400 final-year secondary education students allocated to either an experimental group receiving GoaleTest-based guidance or a control group receiving traditional guidance, using cluster assignment at classroom or school level to minimize contamination. At baseline, all participants will complete the RIASEC questionnaire embedded in the GoaleTest platform, which generates a career affinity score for each available undergraduate program. Primary outcomes are Career Satisfaction (CS), assessed at 6 and 12 months via a purpose-designed 10-item self-report scale, and Academic Retention Index (ARI), operationalized as continued enrollment confirmed through institutional records at 6 and 12 months. Covariates include socioeconomic status, prior academic performance, and learning environment. Internal consistency of the RIASEC instrument will be evaluated using Cronbach’s alpha. Criterion-related, predictive, convergent, discriminant, and ecological validity will be assessed using correlation analyses, group comparisons (t-tests/ANOVAs), logistic regression, and sensitivity analyses. Statistical analyses will be conducted using R version 4.4.1 and IBM SPSS Statistics version 30. Expected Outcomes It is hypothesized that higher GoaleTest affinity scores will be significantly associated with higher career satisfaction (H1), that students with affinity scores above the upper tertile will show significantly lower dropout rates during the first academic year compared to lower-affinity students and the control group (H2), and that integrating longitudinal academic performance data will improve the algorithm’s predictive accuracy through adaptive recalibration of career-specific weightings (H3). This study aims to provide empirical evidence on the validity and educational utility of AI-based vocational guidance systems and to inform the development of scalable, transparent orientation tools for higher education.
Corresponding author: Avilene Rodríguez Lara Competing interests: No competing interests were disclosed.
Grant information: The author(s) declared that no grants were involved in supporting this work.
Copyright: © 2026 Martí-Escolano MÁ and Rodríguez Lara A. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite: Martí-Escolano MÁ and Rodríguez Lara A. Protocol for evaluating the efficacy of GoaleTest, an AI-based vocational guidance system using RIASEC and Nonlinear Vector Alignment [version 1; peer review: awaiting peer review]. F1000Research 2026, 15:1313 (https://doi.org/10.12688/f1000research.186798.1) First published: 06 Aug 2026, 15:1313 (https://doi.org/10.12688/f1000research.186798.1) Latest published: 06 Aug 2026, 15:1313 (https://doi.org/10.12688/f1000research.186798.1)
Advances in digital technologies are driving profound transformations in educational systems, redefining not only how teaching and learning occur but also how students are supported in making academic and career decisions. In particular, the integration of Artificial Intelligence (AI) has enabled the development of tools that deliver personalized recommendations, enhance decision-making processes, and reduce uncertainty in complex educational contexts.1,2
In the domain of vocational guidance, traditional approaches have been criticized for their limited personalization and reliance on static assessments. In contrast, AI-based systems, including recommender systems and predictive analytics models, can integrate multiple data sources such as psychological traits, academic performance, and contextual variables to provide more accurate and individualized guidance.3–5
Learning Analytics has emerged as a key field supporting this transformation, enabling the analysis of learner data to optimize educational outcomes and inform decision-making processes.6 Recent studies have highlighted the potential of AI-driven systems to improve student retention, engagement, and academic alignment, particularly when combined with psychometric profiling.7,8
The present protocol focuses on GoaleTest, a digital vocational guidance system grounded in Holland’s RIASEC model, which posits that satisfaction and performance depend on the congruence between personality traits and environmental characteristics.9 The system employs a nonlinear vector alignment algorithm that incorporates mechanisms for signal and antisignal discrimination to match student profiles to academic programs.
The primary objective of this study is to evaluate the effectiveness and predictive validity of GoaleTest in real educational contexts. Specifically, the study aims to assess its impact on career decision-making, satisfaction, and academic retention, contributing to the growing body of research on AI-driven educational support systems.
GoaleTest is based on Holland’s RIASEC model, a widely validated framework in vocational psychology that links personality traits with occupational environments.9 Previous research has demonstrated that congruence between individual profiles and academic or professional environments is associated with higher satisfaction, persistence, and performance.10
The system integrates this theoretical foundation with Learning Analytics approaches, which enable the transformation of learner-generated data into actionable insights for educational decision-making.6 The six dimensions evaluated are:
• R (Realistic): practical orientation, physical environments, and tools.
• I (Investigative): analysis, abstraction, and intellectual curiosity.
• A (Artistic): creativity, original expression, and open environments.
• S (Social): human interaction, helping, and teaching.
• E (Enterprising): leadership, influence, and goal achievement.
• C (Conventional): structure, accuracy, and data management.
The system integrates this theoretical framework with principles of Learning Analytics, transforming psychometric responses into predictive profiles of academic success to generate personalized undergraduate program recommendations.
GoaleTest is a browser-based digital tool designed for high school and pre-university students.
The undergraduate program database contains a catalog of over 90 programs, classified by subject area and RIASEC profile requirements.
The system consists of:
• Student-Accessible Web Application Interface; RIASEC-Classified Database: 90+ Undergraduate Programs
• Proprietary Algorithm for Career Affinity Computation
• Analysis Engine: Proprietary Nonlinear Vector Alignment Algorithm with Activation Functions and Signal/Antisignal Adjustment Logic.
The study will use a RIASEC questionnaire with Likert-scale responses from 1 to 5 and dimensional weighting, embedded within the GoaleTest platform. Career Satisfaction (CS) will be assessed using a purpose-designed 10-item self-report scale measuring perceived fit between the recommended academic pathway and the student’s own interests and expectations, rated on a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree). The internal consistency of this scale will be evaluated using Cronbach’s alpha prior to main analyses. Academic Retention Index (ARI) will be operationalized as continued enrollment confirmed through institutional records at 6 and 12 months. Covariates will include socioeconomic status (assessed via a sociodemographic questionnaire), prior academic performance (obtained from school records), and learning environment (assessed via a student self-report questionnaire).
Input Normalization
SeaA=a1,a2,…,an represents the set of responses.1–4,6 The normalized score for each dimension D∈{R,I,A,S,E,C} is calculated as:
SD=round(∑(ai·wi)∑wi·5×100)
where wi represents the weight assigned to item i for dimension D. This step places all six dimensions on the same scale, which makes them directly comparable. In educational terms, the transformation converts heterogeneous questionnaire responses into a standardized vocational profile.
Illustrates the architecture of the affinity calculation process; see Figure 1.
To identify the signal corresponding to each program C, a subset of “signal” dimensions IC is defined (the three dominant dimensions of the degree program). Raw similarity is calculated only over these dimensions:
RawSimilarityC=∑k∈ICSk·WC,k∑k∈IC(WC,k)2/100
This equation estimates how closely the student’s strongest vocational traits match the academic profile of a specific degree. Restricting the calculation to the three dominant dimensions reflects the idea that most programs are defined by a small number of core characteristics rather than by all six dimensions equally.
Antisignal Penalization.
A penalty is applied when a student scores high (Sk>45) in a dimension that is irrelevant or counterproductive for the target program dimension (WC,k<30) :
PC=∑k∈AntiSignal(Sk−4555)·MaxPenaltyk
This term is important because a strong trait is not always beneficial if it conflicts with the demands of the chosen program. Educationally, the antisignal component helps the model avoid recommending options that appear attractive at first glance but may not support persistence or satisfaction in practice.
Final Affinity Score
To improve the readability and realism of the results and avoid the 99% “ceiling” effect, a power function is applied:
Fc=(RawSimilarity)1.6.0.88
A linear adjustment is applied according to the combined profile and the program family ± 4%
FinalAffinityC=clamp(10,88,FC−PC+Bonusarea)
The final score summarizes the overall compatibility between the student and the program. It is the output used to compare options and to test whether stronger fit predicts later satisfaction and retention.
GoaleTest operationalizes vocational fit by quantifying the congruence between a student’s RIASEC profile and the RIASEC signature of each academic program. The mathematical steps are not only computational, but also educational: normalization creates comparability, vector alignment captures fit, and antisignal penalization captures mismatch. Together, these elements are intended to connect psychometric compatibility with educational outcomes, such that greater affinity should correspond to greater satisfaction and persistence, and lower dropout.
A quasi-experimental design will be used, with an experimental group receiving GoaleTest guidance and a control group receiving traditional guidance. Assignment to conditions will be conducted at the classroom or school level (cluster assignment), with intact groups allocated to each condition to minimize contamination between groups. Baseline equivalence between conditions will be assessed on key demographic and academic variables prior to analysis. The study will examine whether career affinity is associated with career satisfaction, whether high-affinity students show better retention, and whether the algorithm improves when longitudinal data are incorporated.
Sample size estimation was based on medium effect sizes reported in the educational predictive analytics literature, using conventional assumptions for power analysis in applied educational research.
A preliminary power analysis was conducted using G*Power 3.1 to estimate the minimum sample size required for detecting medium effect sizes in correlational and group-comparison analyses. Assuming an effect size of f = 0.25, a statistical power of 0.80, and a significance level of α = 0.05, the estimated minimum sample size ranged between 180 and 250 participants depending on the statistical model employed. The proposed sample size of 200–400 participants was therefore considered sufficient to ensure adequate statistical power for the planned analyses.
• Final-year secondary education.
• Random cluster sampling.
The GoaleTest implementation utilizes a heuristic, deterministic framework. This framework relies on expert-defined rules and predefined weights to estimate career affinity. This approach ensures early-stage interpretability and transparency. However, the system design facilitates evolution toward a predictive machine learning (ML) architecture.
Future development incorporates a supervised learning approach. This approach pairs RIASEC-based input vectors with longitudinal outcome data, including academic performance, persistence, and retention. This dataset enables training of predictive models, specifically Deep Neural Networks (DNNs), to identify patterns of successful academic trajectories.
Model parameters, including career-specific vectors WC, are optimized using gradient-based methods (e.g., stochastic gradient descent) to improve predictive accuracy. In addition, a recommender-system layer integrates using collaborative filtering techniques. This integration assumes comparable academic outcomes from similar student profiles.
This hybrid architecture transitions from rule-based recommendations to data-driven personalization. It maintains interpretability during initial deployment phases.
The current implementation incorporates heuristic parameters, including the non-linear exponent (1.6) and the affinity clamping range (10–88). Because these parameters were established through expert-driven calibration rather than empirical optimization, a sensitivity analysis will be conducted to evaluate the robustness of the algorithm under alternative parameter configurations.
The analysis will assess the stability of affinity scores across variations in:
• Non-linear exponent values.
• Clamping intervals.
• Antisignal penalty thresholds, and
• Area bonus adjustments.
Changes in score distributions and predictive consistency will be examined to identify parameter configurations that maximize stability and interpretability.
Three primary research questions and associated hypotheses guide the validation:
RQ1/H1 (Decision effectiveness): Is there a significant positive correlation between the affinity scores generated by the algorithm and career satisfaction reported by students six months after program enrollment? It is hypothesized that higher affinity scores will be significantly associated with higher career satisfaction (H1).
RQ2/H2 (Predictive power): Do students with high affinity scores show lower dropout rates during the first academic year compared to students with low affinity scores and to the control group? It is hypothesized that students above the upper tertile of affinity scores will show significantly higher retention rates (H2).
RQ3/H3 (Adaptive segmentation): How do the algorithm’s career-specific weightings (WC) evolve when longitudinal academic performance data are incorporated? It is hypothesized that integrating outcome data will improve the predictive accuracy of the model (H3).
Dependent Variables
The primary dependent variables are the Academic Retention Index (ARI) and Career Satisfaction (CS).
Career Satisfaction (CS) will be assessed through self-reported measures evaluating perceived alignment between the recommended academic pathway and the student’s expectations and interests. An overview of all study variables, measurement instruments, and assessment timepoints is provided in Table 1.
Academic Retention Index (ARI) will be operationalized as continued enrollment and persistence during the first academic year. Conversely, dropout will be operationally defined as formal withdrawal or non-enrollment during the first academic year following enrollment in higher education.
Procedure
The validation process will assess multiple dimensions of reliability and validity. Internal consistency of the RIASEC instrument will be evaluated using Cronbach’s alpha.11 Criterion-related validity will be examined through correlations between career affinity scores and outcome variables, including Career Satisfaction (CS) and Academic Retention Index (ARI).
Predictive validity will be assessed using regression models to evaluate the relationship between affinity scores and student retention, in line with prior research on predictive analytics in education.7 Convergent and discriminant validity will be explored by analyzing the alignment between RIASEC profiles and recommended academic pathways.10
Ecological validity will be evaluated through real-world implementation, assessing system performance in authentic educational contexts over time.
The overall study workflow and follow-up structure are summarized in Figure 2.
Preliminary analyses will examine the correlation between affinity and career satisfaction, as well as t-tests or ANOVAs comparing high- versus low-affinity groups. Predictive analyses will use logistic or multinomial regression to model dropout. Longitudinal analyses will assess the algorithm’s adaptive adjustment using academic performance data. Reliability will be assessed using Cronbach’s alpha, and sensitivity analyses will test alternative exponent values and normalization ranges to examine robustness. Statistical analyses will be conducted using R version 4.4.1 (R Foundation for Statistical Computing, Vienna, Austria, 2024) and IBM SPSS Statistics version 30 (IBM Corp., Armonk, NY, USA, 2023). The statistical analyses planned for each research question are summarized in Table 2.
The study is expected to show that higher career-affinity scores are associated with greater career satisfaction and lower dropout rates over time. It is also anticipated that the longitudinal incorporation of academic outcome data will improve the system’s predictive performance and refine the weighting of career vectors across degree programs.
The planned analyses will allow comparisons between high- and low-affinity groups, as well as between the intervention and control groups, on satisfaction, persistence, and retention. Correlation analyses, regression models, and group comparisons are expected to provide evidence on whether the current rule-based model performs adequately and whether the predictive architecture adds measurable value.
If the model behaves as expected, the results will support the feasibility of combining psychometric profiling with adaptive algorithmic refinement for personalized academic guidance.
This protocol assumes that vocational fit can be operationalized by aligning psychometric profiles with academic environments. GoaleTest is grounded in a well-established principle in vocational psychology: congruence or compatibility between students’ personality characteristics and their academic environment is associated with higher satisfaction and persistence.12 Recent longitudinal research confirms that person-environment fit remains a significant predictor of educational trajectories, with students’ perceived alignment in secondary education indirectly predicting educational pathways through self-concept and motivational beliefs.12 Contemporary approaches to vocational guidance emphasize informed choice, academic-career fit, and reduced information asymmetries to strengthen goal clarity and student-career alignment.10,13
In that sense, the platform does not differ conceptually from standard college planning software, but it extends this framework through a computational model designed to measure and determine compatibility at greater precision and scalability. The proposed framework combines a transparent, rule-based system with a future path toward machine learning, balancing interpretability and scalability. One of its main strengths is the way it handles incompatible combinations: rather than overweighting only the dominant student characteristics, the “antisignal” component explicitly accounts for true mismatches between the student profile and the academic environment, a limitation of many traditional methods. The current threshold values and cut points used in this logic should be understood as provisional and will need empirical confirmation through the present study and future validation work. This approach aligns with recent advances in educational data mining and machine learning for student performance prediction, in which explainability is increasingly recognized as essential for building stakeholder trust and ensuring the ethical deployment of AI systems in educational settings.14–16 Another important advantage of GoaleTest is its interpretability. In contrast to many neural-network-based recommender systems, the platform’s deterministic logic allows educators, counselors, and institutional reviewers to inspect and revise the results without requiring machine-learning expertise. This feature is especially relevant in educational settings where there may be resistance to “black box” systems from counseling staff or ethics committees, and it aligns with broader concerns about transparency in AI-assisted education.14,15 Research on explainable AI in educational contexts demonstrates that transparent systems significantly enhance trust, engagement, and learning outcomes, with user trust levels increasing substantially when explanations of algorithmic decisions are provided.14,15,17,18 Furthermore, educational recommender systems with varying levels of user control and transparency are strongly correlated with perceived transparency and user satisfaction.19 If validated, the tool may contribute to more efficient and equitable academic guidance, particularly in settings where individualized counseling resources are limited. Its educational value lies not only in recommending programs, but also in reducing information asymmetries and supporting more informed decision-making at scale. In that sense, GoaleTest may be especially useful for expanding access to guidance services in contexts where students have little direct support during the transition to higher education. Recent studies on precision employment guidance and career maturity demonstrate that individualized, data-driven approaches significantly enhance student decision-making abilities and employment satisfaction, with customized guidance outperforming traditional one-size-fits-all methods.20 Additionally, research on educational equity in lowresource contexts emphasizes how transparent AI systems, when properly designed with community engagement, can support more equitable access to career guidance and improve educational outcomes.21 However, the interpretation of future findings will depend on sample representativeness, data quality, and the extent to which longitudinal educational outcomes can be reliably captured. Additional limitations may include the need for external validation across institutions and the possibility that model performance varies across socioeconomic or disciplinary contexts. Research on predictive models in education highlights that while machine learning significantly outperforms traditional methods, careful attention must be paid to potential biases in training data, generalizability across diverse student populations, and the need for transparent, equity-centered evaluation frameworks.22,23 For that reason, the present protocol should be understood as a stepwise evaluation of a heuristic guidance engine that may later evolve into a data-driven predictive system, rather than as a final claim of generalizable effectiveness. This iterative approach reflects best practices in the design of AI systems for education, where balancing interpretability with advancing predictive accuracy is critical.13 Contemporary frameworks emphasize moving beyond simple accuracy metrics toward comprehensive evaluation frameworks that assess transparency, fairness, and user-centered design principles.24
Overall, the protocol supports a gradual transition from rule-based vocational guidance to an adaptive, predictive model, providing a methodological basis for future refinement and broader educational application. This approach is consistent with contemporary research on human-centered AI in education, which demonstrates that systems designed with strong interpretability foundations are more likely to gain stakeholder trust and achieve sustainable adoption in educational settings.14,24 By grounding the development of GoaleTest in established vocational psychology principles while leveraging computational advances, this protocol contributes to the growing body of evidence supporting the use of transparent, personalized guidance systems to enhance educational equity and student success.21,25
This protocol outlines a structured evaluation of GoaleTest as an AI-assisted vocational guidance system grounded in the RIASEC model and a rule-based career-affinity algorithm. The proposed study is designed to assess its psychometric validity, predictive utility, and educational applicability in real academic settings. In addition, the protocol establishes a methodological basis for the progressive transition from a heuristic-deterministic approach toward a predictive machine learning architecture. If validated, the system may offer a scalable, personalized strategy for academic guidance, help reduce mismatch-related dropout, and improve decision-making in higher education.
The results of this study will be submitted for publication in a peer-reviewed journal in the fields of educational technology or educational psychology. Preliminary findings will be presented at relevant national and international conferences. All anonymized data generated during the study will be deposited in an open-access repository (e.g., Zenodo or OSF) upon study completion, in compliance with F1000Research data sharing policies. The study protocol itself is being published to allow pre-registration of methods and to facilitate transparency and reproducibility.
The study is currently in the protocol design and ethics review phase. Participant recruitment has not yet commenced. Data collection is expected to begin in January 2027, following institutional ethics approval. The planned completion date for the data collection phase is September 2027.
Ethics approval for this study is currently under review by the Ethics Committee of the International University of Miami (UNIMIAMI) and will be obtained prior to the initiation of participant recruitment. The study will be conducted in full accordance with the principles of the Declaration of Helsinki. Informed consent will be obtained from all participants prior to data collection; for participants under 18 years of age, consent will also be obtained from parents or legal guardians. No participant recruitment or data collection will commence until ethics approval is formally granted.
The GoaleTest career-affinity algorithm is a proprietary system currently under active commercial development; full source code cannot be made openly available at this time. The algorithmic specifications, mathematical formulations, and parameter definitions are described in full detail in the Methods section of this protocol to ensure methodological transparency and reproducibility of the validation study. As this manuscript describes a study protocol and data collection has not yet commenced, the statistical analysis scripts are currently under preparation. Upon completion of the study, all analysis code (R version 4.4.1) will be deposited in a public repository (GitHub) and archived with a persistent identifier (Zenodo DOI). The repository URL and DOI will be included in the final results article. Researchers wishing to obtain further information about the GoaleTest platform may contact the corresponding author at [email protected].
No data are associated with this article. This manuscript describes a study protocol; data collection has not yet commenced pending ethics approval. Upon completion of the study, all anonymised datasets will be deposited in an open-access repository (Zenodo, https://zenodo.org) under a CC-BY licence and will include the raw RIASEC scores, affinity score outputs, career satisfaction measures, and academic retention records. The DOI of the dataset will be included in the final results article.
The authors wish to thank the International University of Miami (UNIMIAMI) for their institutional support during the development of this protocol.
The author(s) declared that no grants were involved in supporting this work.
© 2026 Martí-Escolano MÁ and Rodríguez Lara A. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Current Reviewer Status:
AWAITING PEER REVIEW
AWAITING PEER REVIEW
?
Key to Reviewer Statuses VIEW HIDE
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions