Evaluating the Bibit App: The HEART Framework Approach in UX Design

With the rise of investment applications like Bibit in Indonesia, evaluating user experience is crucial to improve user engagement and satisfaction. This study aims to assess the Bibit app user experience using the HEART framework to identify areas for improvement. A descriptive survey was conducted using the Slovin method to determine a sample size of approximately 100 people. The study used a questionnaire distributed through Google Forms that measures the five aspects of the HEART framework on a Likert scale. Data were analyzed for validity and reliability using IBM SPSS and stored in Google Spreadsheets. Overall, the Bibit app demonstrated high satisfaction, engagement, adoption, retention, and task success among its users, with most categories scoring above the 80% target. However, specific areas, particularly Happiness 2, Adoption 1, Retention 1, and Task Success 3, require further development attention. The Bibit app provides a generally satisfying user experience, with significant potential for improvement in navigation tooltips, investment product development, and app speed and efficiency.


Introduction
In the 21st century, technological advancements have significantly transformed daily life, especially convenience [1]- [3].A prime example is the widespread internet penetration and the use of smartphones, which have become an integral part of life in Indonesia [4]- [6].According to Kemp's 2022 data, there were 204.7 million internet users and 307.1 million smartphones in Indonesia at the start of 2022 [7].This development has not only facilitated communication and information access but has also expanded into sectors like investment [8], [9].Technology enables online investment, as reflected in the popularity of investment apps like Bibit, founded in 2019 by Wellson Lo [10].With this ease of access, there has been a significant increase in the number of local stock investors in Indonesia, exceeding 4 million, according to the first semester of 2022 data from KSEI [11].However, this growth brings new challenges, particularly regarding user experience in investment apps.
Mobile applications designed to run on mobile devices simplify data processing and task execution.These apps are specifically tailored for mobile devices and add various functionalities [12]- [15].Besides ease of use, mobile applications offer downloadable and installable features, providing accessible and customizable options for users [16]- [18].Their importance is evident in everyday life, offering practical solutions for diverse needs, from communication to task management.In terms of application performance analysis, this involves assessing the app from various aspects, including user interface (UI) and user experience (UX) [19]- [21].
UX evaluation is crucial in determining an app's success, as investigated in Bibit [22].UX focuses on user impressions of the product, ensuring that the app meets functional needs and provides a pleasant and intuitive experience [23].Google's HEART Framework is also vital for measuring aspects like Happiness, Engagement, Adoption, Retention, and Task Success about user experience, offering guidance for identifying goals, measuring user behavior, and setting relevant metrics for app performance improvement [24]- [26].
Previous studies have examined the Bibit app.For instance, the study by Irawan & Sfenrianto, [27] evaluated user experience in investment apps, similar to this research.Both studies focus on the usage of online investment apps, with Irawan & Sfenrianto, [27] exploring factors influencing investors' intentions to use mutual fund investment apps.This is akin to the study by Putra et al., [28] which also concentrates on user retention in investment apps.Both studies assess aspects affecting user satisfaction and the sustainability of app usage.
Additionally, this research focuses on the study by Handoko & Mozes, [29] which examines investment app usage.Both studies employ quantitative methods to analyze factors influencing app usage.However, Handoko & Mozes, [29] used the Technology Acceptance Model (TAM) specifically to assess Bibit.The differences among these studies are notable.Irawan & Sfenrianto, [27] did not use the HEART framework in their analysis, focusing more on perceived ease of use and trust to influence user intentions, without exploring happiness, engagement, adoption, retention, and task success.
Similarly, Putra et al., [28] while also focused on user retention, did not adopt the HEART Framework approach used in this study.Furthermore, their study is based on the expectation-confirmation model (ECM) and emphasizes user satisfaction, perceived usability, and enjoyment, whereas this study on Bibit considers a broader range of variables.Lastly, this research differs in its specific focus on the Bibit app and the use of a modified TAM, investigating diverse aspects such as financial literacy and risk perception, aspects not directly explored in the study by Handoko & Mozes, [29] with the HEART framework.
Therefore, this study aims to evaluate the user experience of the Bibit app using the HEART Framework and to understand the direction of Bibit app development.Hence, the research questions are as follows: What are the results of assessing the user experience of the Bibit app using the HEART Framework, and what categories require further improvement?Thus, the novelty of this research lies in the unique application of the HEART Framework in the context of investment app evaluation.This framework, which encompasses Happiness, Engagement, Adoption, Retention, and Task Success, provides a holistic and diverse perspective in measuring user experience.
It allows the study to evaluate functional factors like usability and ease of use and consider emotional aspects and the user's engagement level with the app.This approach offers deep insights into how the Bibit app is received and used by users, providing more comprehensive guidance for future app development compared to previous studies, which might focus only on specific aspects like user intention, satisfaction, or technical factors.

Research Methods
The research method used in this study is a descriptive survey, focusing primarily on evaluating the user experience of the Bibit app using the HEART Framework.The HEART, developed by the Google team, encompasses five categories: Happiness, Engagement, Adoption, Retention, and Task Success.This was chosen because it provides a structured framework for identifying goals and measuring success, focusing on user needs.Using HEART, this study aims to support product development decision-making for Bibit based on user needs, highlight areas requiring improvement, and ensure the app delivers the best user experience.
The HEART framework does not focus on specific metrics.Still, it offers a structured way to organize metrics that should be captured in the evaluation to ensure that all useful aspects have been covered.In this study, the researchers developed a set of main themes for Goal-Signal-Metrics to obtain an assessment of the Bibit app's user experience, illustrated as seen in Table 1.The sampling technique applied is Slovin's method, which effectively determines the sample size from a large population.The sample size calculation uses Slovin's formula as seen in Formula 1.
n represents the sample size or the number of respondents.N is the size of the population, and e is the percentage of error that can be tolerated in the research.From this data, the population size is 5 million downloads.In this study, a 10% error percentage is also used.Following the calculation, the sample size was 99,998, rounded to 100 respondents.
The instrument used in this research is a questionnaire distributed via Google Forms.This questionnaire consists of questions designed to measure the five aspects of the HEART Framework, including Happiness, Engagement, Adoption, Retention, and Task Success.These questions are intended to be answered using a Likert scale, ranging from 1 (Strongly Disagree) to 5 (Strongly Agree), to measure the level of satisfaction and user experience with the Bibit app.
Based on the questionnaire results conducted and entered into a Google Spreadsheet, their validity, and reliability are tested using IBM SPSS software.Finally, data processing is carried out using Likert scale calculation analysis.After calculating the obtained data, conclusions are drawn from the results according to the predetermined Goal-Signal-Metrics.For data analysis techniques, after the data is collected, analysis is performed by calculating the total score of respondents' answers using Formula 2.

𝐹𝑜𝑟𝑚𝑢𝑙𝑎 = 𝑇 𝑋 𝑃𝑛
(2) T is the Total number of Respondents who voted, Pn is the Selected Likert Number Then, an interpretation of scores is conducted to determine the level of respondents' agreement with each question item.This process involves calculating the highest score (Y), as in Formula 3.
Next is the determination of interpretation intervals, as seen in Formula 4.
Interval's Formula: The percentage index calculation is then categorized into predetermined ranking categories as seen in Formula 5.

Results
After the questionnaire distribution, 100 respondents filled out the questionnaire.The data obtained was processed for comparison with predetermined objectives.This processed data was used to evaluate the Bibit app, determining whether the app needs further development and identifying the factors that require improvement.The questionnaire was distributed from April to May 2023 and was completed by 100 respondents who have used the Bibit app.Most data was obtained from users of the Bibit app for mutual fund investments, amounting to 46 respondents.The data visualization can be seen in Figure 1.The respondents came from various backgrounds, which can be seen in Figure 2 of these 100 respondents, 45% were private employees, followed by 21% entrepreneurs, 15% teachers, lecturers, or educational  Validity and reliability tests were then conducted on the questionnaire results to verify their validity and reliability.These tests were conducted using the IBM SPSS application, commonly used for statistical purposes.The R-Table used was for 100 respondents, and the significance level was for a two-tailed test.The testing results showed that each category in the survey was valid with excellent scores, as the validity test results for each category were well above the R-table values, and all categories were declared valid.Reliability testing was then continued.The Cronbach's alpha value sees a good questionnaire reliability benchmark.If it is > 0.9, then the reliability is perfect; if between 0.7 -0.9, then it is high; if between 0.5-0.7,then it is moderate; and if below 0.5, then it is low is shown in Table 2.A low questionnaire reliability means that one or many items are not reliable.The reliability test results showed that the questionnaire had perfect reliability as it had a Cronbach's alpha of more than 0.9.Calculations were also made on the possible Cronbach's alpha values if some items were removed to affirm its perfect reliability.
The research used a questionnaire to measure five categories: Happiness, Engagement, Adoption, Retention, and Task Success, with 100 respondents providing interesting findings.Most respondents gave high ratings in the Happiness category, divided into five subcategories (H1-H5).Specifically, H1 reached a score of 88.2%, and H3 and H4 also showed very good results with scores of 87% and 86.2%, respectively.H5 also exceeded the research target with a score of 83.6%, indicating high satisfaction of respondents with the aspects measured.
The Engagement category (E1-E4) also showed positive results, with all subcategories exceeding the research target of 80%.E1 reached a score of 83%, E2 rose to 85.4%, E3 recorded 82%, and E4 reached 84.8%.This indicates a high level of respondent engagement with the aspects measured in the questionnaire.In the Adoption category A1 and A2, the results were varied.A1 approached the research target with a score of 79.8%, slightly below the 80% threshold, while A2 exceeded the target by 82.2%.This indicates that some aspects in the Adoption category still require attention and further development.
Most subcategories for the Retention category, consisting of R1 to R5, successfully exceeded the research target.R1 recorded a score of 79%, slightly below the target, but R2, R3, R4, and R5 each recorded scores of 82.2%, 85.6%, 82.8%, and 83.4%, indicating that the majority of respondents have the desire to continue using or participating in the aspects measured.3. Which categories still fail to meet the target?.
To improve performance in the four categories that have not reached the 80% target, here are specific development recommendations for each category: Happiness 2: In this category, the score obtained is still below the 80% target.As this category relates to the ease of use of the Bibit app, it is suggested that tooltips be added to its features.These tooltips will help users understand the function of each feature.It is also important to communicate this information through social media and advertisements so users are not confused when using the Bibit app.
Adoption 1: This category requires further development in the Bibit app, especially regarding investment products.The survey results indicate that users' investment needs are not being met.Therefore, the development of investment products in this app is very necessary.Adding a comparison feature that displays data and growth charts for each investment product is also important, allowing users to make better decisions and not switch to competitor apps.
Retention 1: This category has not met the target and shows the need for app development according to feedback from the Retention 1 questionnaire.Development suggestions include adding features that make it easier for users to choose the best mutual fund products, such as a comparison feature that can compare several mutual fund products.This will help users feel more comfortable and continue using the Bibit app for their mutual fund investments.
Task Success 3: In this category, the primary focus is on the speed of the Bibit app.Developers are advised to improve transaction processing speed and loading of features or content.Some ways to implement this include effectively caching data for faster app loading, optimizing image size to reduce memory usage, and reducing initial app loading time, for example, by using a splash screen.These areas will enhance the overall user experience and the app's efficiency.
By implementing these recommendations, it is hoped that each category will reach or exceed the 80% target, thus increasing overall user satisfaction and retention of the Bibit app.

Discussions
A user experience evaluation of the Bibit application, employing the HEART framework, has revealed significant findings.In the Happiness dimension, most respondents expressed satisfaction, particularly in the subcategory H1, which scored 88.2%.This indicates the application's success in providing a pleasant user experience.For Engagement, all subcategories exceeded the research target of 80%, indicating high user involvement with the application.
However, the Adoption category showed varied results, with A1 slightly below the research target, suggesting the need for improvement in certain aspects.In Retention, most subcategories showed positive results, except for R1, which was slightly below target, indicating the necessity of enhancing specific aspects to boost continuous user engagement with the application.Lastly, Task Success revealed achievements in most aspects, but T3 fell below the target, requiring additional focus.Overall, this evaluation highlights the importance of continuous development to meet and exceed user expectations.
Findings from the Bibit application's user evaluation are supported by previous studies that have utilized the HEART framework.These studies emphasize the significance of the five aspects within the HEART framework in understanding and enhancing user experience.For instance, research conducted by Putra et al., [28] and Ellitan et al., [30], [31] demonstrated that user satisfaction (Happiness) directly affects user retention, which is consistent with findings in the Bibit application.Similarly, other research conducted by Nurjanah et al., [32] and Hanif et al., [33] confirmed that user engagement (Engagement) is key to building loyalty and enhancing product adoption.Research focusing on Adoption emphasized that a deep understanding of user needs can strengthen product adoption, as reflected in the varied scores in the Adoption category in the Bibit application.Further, studies related to Retention and Task Success, as explained by Alqahtani & Orji [34], show these aspects are crucial in maintaining user interest and application efficiency.These collective findings affirm the results from the Bibit application evaluation, underlining the importance of each dimension in the HEART framework for a holistic understanding and improvement of user experience.
Based on the questionnaire analysis and descriptive evaluation, four main categories in the Bibit application require improvement to achieve user satisfaction targets.First, in the Happiness 2 category, enhancing ease of use is necessary, which could be achieved by adding tooltips to application features to help users understand each feature's function.Second, in the Adoption 1 category, enhancing investment products is key, where the application needs to provide clearer information and comparisons between investment products to aid users in making more informed decisions.
Third, for the Retention 1 category, the application should develop features that ease the selection and comparison of mutual fund products, thus increasing user satisfaction and retention.Lastly, in the Task Success 3 category, the application needs to improve transaction processing speed and content loading, which is achievable through technical optimization like data caching and reducing initial loading times.Implementing these development strategies is expected to enhance overall user satisfaction and retention.
The findings and development recommendations for the Bibit application are significantly aligned with prior research in application development and user satisfaction.For example, research by Prastyo et al., [31] and Tirana & Sfenrianto [35] emphasized the importance of ease of use in mobile applications as a primary factor in enhancing user satisfaction, aligning with the recommended improvements in the Happiness 2 category of the Bibit application.Likewise, studies conducted by Vatanasakdakul et al., [36] and Hoang & Le Tan, [37] on user behavior in technology product adoption showed that information transparency and ease of product comparison can increase trust and user satisfaction, supporting recommendations for the Adoption 1 category.
Further, research by Rizvanović et al., [39] and Dwivedi et al., [40]  This research provides valuable insights into current user needs and preferences and sets a foundation for continuous innovation and improvement in the Bibit application to meet and exceed user expectations.

Conclusions
The conclusion of this study indicates that the Bibit application generally provides a satisfying experience to its users, particularly in aspects of happiness and engagement, with the majority of subcategories surpassing the research targets.However, there is room for improvement, especially in the categories of Happiness 2, Adoption 1, Retention 1, and Task Success 3, which have not yet reached the 80% target.Practical recommendations for the Bibit application include the addition of tooltips to facilitate navigation and understanding of features, the development of investment products more aligned with user needs, and enhancements in the speed and efficiency of the application.These suggestions are aimed at the developers of the Bibit application to improve user satisfaction and retention and encourage broader adoption.This research contributes significantly to the understanding of factors that influence user satisfaction in financial applications, and its results can be used as a reference for the development of similar applications in the future.

Figure 1 .
Figure 1.Percentage of investment products used by respondents in the Bibit app

Table 1 .
Goal-Signal-Metric used in the UX evaluation of the Bibit app

Table 2 .
Possible Cronbach's alpha values if an item is removed

Table 3
which used the TAM model to analyze the Bibit app with a focus on aspects such as ease of use and perceived risk, this study integrates a more diverse assessment, including factors such as happiness and engagement, providing broader insights into how the app is received by its users.This approach results in a more holistic understanding of user needs and preferences, enabling more targeted improvements.Therefore, this research has significant implications for developing the Bibit application, particularly in enhancing user satisfaction and retention.The questionnaire results, measuring the five key categories -Happiness, Engagement, Adoption, Retention, and Task Success -highlight areas requiring improvement.Notably, four categories have not met the 80% target: