Data Science
Big Data & Data analysis Survey, University Work
Hi there im currently in university studying Computing, and one of the topics im covering right now is Big data and data analysts. My task is to use at least one method that provides me insights and aswsers in big data and data analysts to conduct my own research into this sector. Ive created a form, i would appreciate if you would fill it out. The survey talks about your understanding of big data and if you use it within your job.
Here is the link to the survey - https://forms.office.com/Pages/ResponsePage.aspx?id=qm3krgo4gEiLZ_QvcCrT8v3CQ52fllpApP5yo39hPI5UMzBFTDlGQTVQVFFFV1pNR1VERkk0WE1GSi4u
For anyone who does fill this form out, thankyou this will help me alot with my university work.
submitted by /u/haroldbaxter
[link] [comments]
Big Data & Data analysis Survey, University Work
Hi there im currently in university studying Computing, and one of the topics im covering right now is Big data and data analysts. My task is to use at least one method that provides me insights and aswsers in big data and data analysts to conduct my own research into this sector. Ive created a form, i would appreciate if you would fill it out. The survey talks about your understanding of big data and if you use it within your job.
Here is the link to the survey - https://forms.office.com/Pages/ResponsePage.aspx?id=qm3krgo4gEiLZ_QvcCrT8v3CQ52fllpApP5yo39hPI5UMzBFTDlGQTVQVFFFV1pNR1VERkk0WE1GSi4u
For anyone who does fill this form out, thankyou this will help me alot with my university work.
submitted by /u/haroldbaxter
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
Machine Learning projects on jupyter
Hello everyone,
Im a recent Data Science graduate and experimenting with different machine learning models on a various datasets from kaggle. The idea is to be more comfortable with tensorflow (& other libaries) and different datasets.
Im doing all this on jupyter notebook, is there a tool data scientist use to publish their work. I want to create a online portfolio which i can showcase the different ML implementations in interviews and to recruiters.
Im using github but was wondering if there specific tools or practices of data scientists which i might aswell implement in my workflow
Thanks
submitted by /u/exodusgg
[link] [comments]
Machine Learning projects on jupyter
Hello everyone,
Im a recent Data Science graduate and experimenting with different machine learning models on a various datasets from kaggle. The idea is to be more comfortable with tensorflow (& other libaries) and different datasets.
Im doing all this on jupyter notebook, is there a tool data scientist use to publish their work. I want to create a online portfolio which i can showcase the different ML implementations in interviews and to recruiters.
Im using github but was wondering if there specific tools or practices of data scientists which i might aswell implement in my workflow
Thanks
submitted by /u/exodusgg
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
What is considered a "high-value" data scientist and how do you become one?
I'm a data scientist with a 2 years of experience. I've realised recently that all of my projects in industry follow the same basic blueprint which I learnt years ago:
1. Think of how to best frame the problem so it can be solved using data science.
2. Clean and process the data.
3. Extract features which give the highest accuracy
4. Sticking it into an sklearn algorithm and hoping it has good accuracy (or whatever metric is important for that specific problem).
5. If it has bad accuracy, redo step 3 or get more data. If it has good metrics, done.
My issue is: anyone who has done a data science boot camp can easily reproduce these steps. Coding in R and Python is super easy, especially nowadays with libraries like sklearn.
* Is a data scientist considered "high-value" because of step 1? Or is it the fact that they have lots of ideas about how something can be modelled which goes beyond the "traditional" models?
* If it is the latter, is the only way to create more complex models to just gain more experience with a variety of problems, and coming across new techniques whilst researching? I find that I don't even know about the existence of many techniques unless it's pointed out to me by colleagues.
By "complex" model. I don't mean complicating the algorithm a model uses for the sake of it. I'm referring to a more robust model that takes more variables into account.
As data scientists get more experienced, in which ways can they provide more value to a business compared to say, an intern, who could probably do all the steps listed above relatively easily?
submitted by /u/Accurate_Green_7001
[link] [comments]
What is considered a "high-value" data scientist and how do you become one?
I'm a data scientist with a 2 years of experience. I've realised recently that all of my projects in industry follow the same basic blueprint which I learnt years ago:
1. Think of how to best frame the problem so it can be solved using data science.
2. Clean and process the data.
3. Extract features which give the highest accuracy
4. Sticking it into an sklearn algorithm and hoping it has good accuracy (or whatever metric is important for that specific problem).
5. If it has bad accuracy, redo step 3 or get more data. If it has good metrics, done.
My issue is: anyone who has done a data science boot camp can easily reproduce these steps. Coding in R and Python is super easy, especially nowadays with libraries like sklearn.
* Is a data scientist considered "high-value" because of step 1? Or is it the fact that they have lots of ideas about how something can be modelled which goes beyond the "traditional" models?
* If it is the latter, is the only way to create more complex models to just gain more experience with a variety of problems, and coming across new techniques whilst researching? I find that I don't even know about the existence of many techniques unless it's pointed out to me by colleagues.
By "complex" model. I don't mean complicating the algorithm a model uses for the sake of it. I'm referring to a more robust model that takes more variables into account.
As data scientists get more experienced, in which ways can they provide more value to a business compared to say, an intern, who could probably do all the steps listed above relatively easily?
submitted by /u/Accurate_Green_7001
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
How far do i need to go in statistics?
I just bought a basic statistics books and it has Exploratory Data Analysis, Probability and Statistical Inference.
I do think this is not enough, what am i missing?
submitted by /u/Business-Chip-7478
[link] [comments]
How far do i need to go in statistics?
I just bought a basic statistics books and it has Exploratory Data Analysis, Probability and Statistical Inference.
I do think this is not enough, what am i missing?
submitted by /u/Business-Chip-7478
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
Portfolio! Help and tips :)
Hi there lovely community!! I’m a DS specialized in oncology research and I have 3 years of experience. I work for a company where everything is confidential so when it comes to build my portfolio I have to start from zero, which is a bad and a good thing at the same time. Bad thing: I don’t have any projects to show but I’ll create some!! Good thing: I can design it from scratch!
I don’t know where to start. I don’t know what to show, what projects to make. I work in a VERY specific area so I can’t even get similar data online.
Also, I’ve never seen a DS portfolio in my life. I was wondering if you have shared yours somewhere or know where I can find examples online.
Even though my portfolio will be related to biology/cancer data, I think looking at some general DS portfolios will help me to brainstorm about mine.
Thank you in advance! Also, any tip will be appreciated!!
submitted by /u/agusconrrero
[link] [comments]
Portfolio! Help and tips :)
Hi there lovely community!! I’m a DS specialized in oncology research and I have 3 years of experience. I work for a company where everything is confidential so when it comes to build my portfolio I have to start from zero, which is a bad and a good thing at the same time. Bad thing: I don’t have any projects to show but I’ll create some!! Good thing: I can design it from scratch!
I don’t know where to start. I don’t know what to show, what projects to make. I work in a VERY specific area so I can’t even get similar data online.
Also, I’ve never seen a DS portfolio in my life. I was wondering if you have shared yours somewhere or know where I can find examples online.
Even though my portfolio will be related to biology/cancer data, I think looking at some general DS portfolios will help me to brainstorm about mine.
Thank you in advance! Also, any tip will be appreciated!!
submitted by /u/agusconrrero
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
Question
Hello, g12 student here looking to apply to a university program, im an Ontario student wanting to apply to a Data Science program and I was wondering if the program and degree is worth it, or whether I should choose something else. What jobs would a bachelor's in Data Science degree open me up to? (ofc I will do my masters). What salary ranges typically are those jobs (Canada and US) and how much do you make as a Data Scientist/ML engineer etc. Is a comp sci program better than a Data Science program? can I get into ML engineering with a data sci degree?
submitted by /u/Antique-Nothing-4315
[link] [comments]
Question
Hello, g12 student here looking to apply to a university program, im an Ontario student wanting to apply to a Data Science program and I was wondering if the program and degree is worth it, or whether I should choose something else. What jobs would a bachelor's in Data Science degree open me up to? (ofc I will do my masters). What salary ranges typically are those jobs (Canada and US) and how much do you make as a Data Scientist/ML engineer etc. Is a comp sci program better than a Data Science program? can I get into ML engineering with a data sci degree?
submitted by /u/Antique-Nothing-4315
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
Star Trek Sega Genesis Data
https://external-preview.redd.it/G8J4hhyRyRiUziupuEdr47FzPjXKsai9ssEM14QGydI.jpg?width=320&crop=smart&auto=webp&s=ef97bcca38da1aa722e84f33e46997c0ffb3b685 submitted by /u/Physical-Building-19
[link] [comments]
Star Trek Sega Genesis Data
https://external-preview.redd.it/G8J4hhyRyRiUziupuEdr47FzPjXKsai9ssEM14QGydI.jpg?width=320&crop=smart&auto=webp&s=ef97bcca38da1aa722e84f33e46997c0ffb3b685 submitted by /u/Physical-Building-19
[link] [comments]
Reddit
From the datascience community on Reddit: Star Trek Sega Genesis Data
Explore this post and more from the datascience community
Data Science
Internship Pros
May be a dumb question: I’ve seen Masters in Data Science/Analytics complain and comment that it took anywhere from 6 to 18 months to get a job having graduated.
Can one expect that time to be shorter if one has had an internship? Anyone have experience with this? How much do employers value an internship?
submitted by /u/ninerniner09
[link] [comments]
Internship Pros
May be a dumb question: I’ve seen Masters in Data Science/Analytics complain and comment that it took anywhere from 6 to 18 months to get a job having graduated.
Can one expect that time to be shorter if one has had an internship? Anyone have experience with this? How much do employers value an internship?
submitted by /u/ninerniner09
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
Streaming Data Observability & Quality
https://b.thumbs.redditmedia.com/Tl3w2xVE57vn8u2sLryaGUO3o5v3SwJpK9gLXRV35bE.jpg We have been exploring the space of "Streaming Data Observability & Quality". We do have some thoughts and questions and would love to get members view on them.
Q1. Many vendors are shifting left by moving data quality checks from the warehouse to Kafka / messaging systems. What are the benefits of shifting-left ?
Q2. Can you rank the feature set by importance (according to you) ? What other features would you like to see in a streaming data quality tool ?
* Broker observability & pipeline monitoring (events per second, consumer lag etc.)
* Schema checks and Dead Letter Queues (with replayability)
* Validation on data values (numeric distributions & profiling, volume, freshness, segmentation etc.)
* Stream lineage to perform RCA
Q3. Who would be an ideal candidate (industry, streaming scale, team size) where there is an urgent need to monitor, observe and validate data in streaming pipelines?
https://preview.redd.it/8f1mo89ouowb1.jpg?width=6998&format=pjpg&auto=webp&s=c1b368112465cfa5be67258dd2a52313cdb4cdb6
submitted by /u/daftpunkapi
[link] [comments]
Streaming Data Observability & Quality
https://b.thumbs.redditmedia.com/Tl3w2xVE57vn8u2sLryaGUO3o5v3SwJpK9gLXRV35bE.jpg We have been exploring the space of "Streaming Data Observability & Quality". We do have some thoughts and questions and would love to get members view on them.
Q1. Many vendors are shifting left by moving data quality checks from the warehouse to Kafka / messaging systems. What are the benefits of shifting-left ?
Q2. Can you rank the feature set by importance (according to you) ? What other features would you like to see in a streaming data quality tool ?
* Broker observability & pipeline monitoring (events per second, consumer lag etc.)
* Schema checks and Dead Letter Queues (with replayability)
* Validation on data values (numeric distributions & profiling, volume, freshness, segmentation etc.)
* Stream lineage to perform RCA
Q3. Who would be an ideal candidate (industry, streaming scale, team size) where there is an urgent need to monitor, observe and validate data in streaming pipelines?
https://preview.redd.it/8f1mo89ouowb1.jpg?width=6998&format=pjpg&auto=webp&s=c1b368112465cfa5be67258dd2a52313cdb4cdb6
submitted by /u/daftpunkapi
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
The test of a first-rate intelligence is the ability to hold two opposed ideas in the mind at the same time, and still retain the ability to function" — F. Scott Fitzgerald. I just realized Barycentric Coordinate System generalizes this "quote idea" to "N" ideas in Ideas Space.
https://preview.redd.it/psw1kuf7wowb1.jpg?width=320&crop=smart&auto=webp&s=31bf556da8e61f6de10ef93d735276e916f8ba56 submitted by /u/Easy_Butterfly2125
[link] [comments]
The test of a first-rate intelligence is the ability to hold two opposed ideas in the mind at the same time, and still retain the ability to function" — F. Scott Fitzgerald. I just realized Barycentric Coordinate System generalizes this "quote idea" to "N" ideas in Ideas Space.
https://preview.redd.it/psw1kuf7wowb1.jpg?width=320&crop=smart&auto=webp&s=31bf556da8e61f6de10ef93d735276e916f8ba56 submitted by /u/Easy_Butterfly2125
[link] [comments]
Reddit
From the datascience community on Reddit: The test of a first-rate intelligence is the ability to hold two opposed ideas in the…
Explore this post and more from the datascience community
Data Science
Is this position normal?
Hey all,
Going to preface this by saying English isn’t my first language so sorry for the bad English if there is any.
So I’m a Junior Data Scientist at a company right now, and I was wondering how normal my situation is. I have around 2 years of experience working with data in my jobs.
I am currently heading an entire (pretty huge) project on my own. One that is pretty high-stakes and will take months (more realistically, years) to get “right”. Everything from data collection, EDA, preprocessing, model creation, and testing has been done and will be done solely by me. This is all for a product that is very consumer facing. Also, it isn’t a straightforward task. It isn’t your classic here’s some data, make some model that makes predictions. It’s a bit more involved than this, which is why this project will take this long to complete.
Is this normal? I do receive very good help from my manager whenever I run into issues, but I’m the only one working on this project. I think I’m doing pretty well with progress but is this normal for a “Junior”? Seems like a lot. I get paid less than 100k, but feeling like I have TONS of responsibility here, and am doing so much work here. I consistently work 60-70 hour weeks to make this happen.
I’m based out of the US, and this is for finance company, if that is relevant.
Thanks!
submitted by /u/Live-Grapefruit2840
[link] [comments]
Is this position normal?
Hey all,
Going to preface this by saying English isn’t my first language so sorry for the bad English if there is any.
So I’m a Junior Data Scientist at a company right now, and I was wondering how normal my situation is. I have around 2 years of experience working with data in my jobs.
I am currently heading an entire (pretty huge) project on my own. One that is pretty high-stakes and will take months (more realistically, years) to get “right”. Everything from data collection, EDA, preprocessing, model creation, and testing has been done and will be done solely by me. This is all for a product that is very consumer facing. Also, it isn’t a straightforward task. It isn’t your classic here’s some data, make some model that makes predictions. It’s a bit more involved than this, which is why this project will take this long to complete.
Is this normal? I do receive very good help from my manager whenever I run into issues, but I’m the only one working on this project. I think I’m doing pretty well with progress but is this normal for a “Junior”? Seems like a lot. I get paid less than 100k, but feeling like I have TONS of responsibility here, and am doing so much work here. I consistently work 60-70 hour weeks to make this happen.
I’m based out of the US, and this is for finance company, if that is relevant.
Thanks!
submitted by /u/Live-Grapefruit2840
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
Just a small question with regards to undergrad module selection
I would greatly appreciate any input since I'm quite stumped on what to do.
So I'm in my final year of undergrad studying a BSc in math and econ, and I have a few optional modules that I'm not sure how I should go about selecting.
For context in my degree I have already covered Advanced linear algebra, multivariate calculus and some topics in discrete math such as number theory, abstract algebra and real analysis.
The modules in question are 1) Advanced statistics (split into two half courses: Distribution theory and statistical inference) 2) Programming for data science 3) Statistical methods for market research and 4) Machine Learning
So based on how my universities has allowed selections, I can either study:
1) Advanced statistics and market research
2) Programming for data science market research
3) Programming for data science and machine learning
Now I am halfway through a python course in udemy (José Portilla's Complete Python bootcamp) and I intend to complete it by this December, and I can also attend the programming lectures even if I don't take the course (which I'm planning to do) and complete the coursework so I can publish it on github. With this in mind, what choice would of modules would be the best? I want to get into DS with a masters course in either applied math/Stats, but I want to work a year after my degree before my masters, hence I want to make sure I go into the job market with some marketable DS skills. Any input is so very appreciated!
submitted by /u/PepperAcrobatic7559
[link] [comments]
Just a small question with regards to undergrad module selection
I would greatly appreciate any input since I'm quite stumped on what to do.
So I'm in my final year of undergrad studying a BSc in math and econ, and I have a few optional modules that I'm not sure how I should go about selecting.
For context in my degree I have already covered Advanced linear algebra, multivariate calculus and some topics in discrete math such as number theory, abstract algebra and real analysis.
The modules in question are 1) Advanced statistics (split into two half courses: Distribution theory and statistical inference) 2) Programming for data science 3) Statistical methods for market research and 4) Machine Learning
So based on how my universities has allowed selections, I can either study:
1) Advanced statistics and market research
2) Programming for data science market research
3) Programming for data science and machine learning
Now I am halfway through a python course in udemy (José Portilla's Complete Python bootcamp) and I intend to complete it by this December, and I can also attend the programming lectures even if I don't take the course (which I'm planning to do) and complete the coursework so I can publish it on github. With this in mind, what choice would of modules would be the best? I want to get into DS with a masters course in either applied math/Stats, but I want to work a year after my degree before my masters, hence I want to make sure I go into the job market with some marketable DS skills. Any input is so very appreciated!
submitted by /u/PepperAcrobatic7559
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
MLOps in Vertex AI in 3 minutes
https://external-preview.redd.it/Bo4avYU_Wdu3bxg2fAW-aoMnbMb8Xbvi1hI7swB0YDw.jpg?width=320&crop=smart&auto=webp&s=063052ffa0344bdebfff82bec76a3fa297f2b17e submitted by /u/fancypigollo
[link] [comments]
MLOps in Vertex AI in 3 minutes
https://external-preview.redd.it/Bo4avYU_Wdu3bxg2fAW-aoMnbMb8Xbvi1hI7swB0YDw.jpg?width=320&crop=smart&auto=webp&s=063052ffa0344bdebfff82bec76a3fa297f2b17e submitted by /u/fancypigollo
[link] [comments]
Reddit
From the datascience community on Reddit: MLOps in Vertex AI in 3 minutes
Explore this post and more from the datascience community
Data Science
TikTok DS Experience
Have an offer from Tiktok for a product DS role. Anyone here at/ was at Tiktok willing to share more about their experience?
The offer is tempting as it’s a +50% bump to my current TC, but I’ve heard that TikTok is pretty hardcore especially for people who don’t speak mandarin so I am looking to gather more info before making a decision.
For context, I don’t speak mandarin and am used to working ~55 hours/week but I do not work weekends.
submitted by /u/slutsky22
[link] [comments]
TikTok DS Experience
Have an offer from Tiktok for a product DS role. Anyone here at/ was at Tiktok willing to share more about their experience?
The offer is tempting as it’s a +50% bump to my current TC, but I’ve heard that TikTok is pretty hardcore especially for people who don’t speak mandarin so I am looking to gather more info before making a decision.
For context, I don’t speak mandarin and am used to working ~55 hours/week but I do not work weekends.
submitted by /u/slutsky22
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
Bayesian modeling vs deep learning
Hi All :)
I’m approaching my final semester in the UIUC online master’s in computer science / data science and was torn between taking bayesian modeling or deep learning as my final elective. As an aspiring data scientist, what would you recommend I take? Is one or the other more beneficial and/or practical to have knowledge in?
To provide some context, I have an undergrad degree in Math, so I’m comfortable with math/stats and have enjoyed some stats courses I’ve taken during my master’s.
One reason for wanting to take bayesian modeling is that there seem to be a lot more accessible resources out there for deep learning intros as opposed to bayesian modeling. However, deep learning is also hot right now, so I’m debating if I should be hopping on the bandwagon.
Any thoughts/advice/suggestions/resources for self-learning would be much appreciated!
submitted by /u/kumquatjk2
[link] [comments]
Bayesian modeling vs deep learning
Hi All :)
I’m approaching my final semester in the UIUC online master’s in computer science / data science and was torn between taking bayesian modeling or deep learning as my final elective. As an aspiring data scientist, what would you recommend I take? Is one or the other more beneficial and/or practical to have knowledge in?
To provide some context, I have an undergrad degree in Math, so I’m comfortable with math/stats and have enjoyed some stats courses I’ve taken during my master’s.
One reason for wanting to take bayesian modeling is that there seem to be a lot more accessible resources out there for deep learning intros as opposed to bayesian modeling. However, deep learning is also hot right now, so I’m debating if I should be hopping on the bandwagon.
Any thoughts/advice/suggestions/resources for self-learning would be much appreciated!
submitted by /u/kumquatjk2
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
How much time do you guys spend in PowerPoint?
I know this question might be a bit controversial to some but lately I've found myself spending an ungodly amount of time creating slides instead of doing other tasks.
Management wants each new project or idea laid out in meticolous detail in a PowerPoint before signing off on it and granting any type of access to data. Which means I need to create some really good looking PPT decks to even get the chance to explore our available data, it's quite frustrating and I'd rather spend the time doing something else.
By detail I mean like, budget, development timeline, target audience, documentation, blabla, before I even get a chance to look at the data and determine if it's useable in the first place.
Anyone else have this problem?
submitted by /u/NipponPanda
[link] [comments]
How much time do you guys spend in PowerPoint?
I know this question might be a bit controversial to some but lately I've found myself spending an ungodly amount of time creating slides instead of doing other tasks.
Management wants each new project or idea laid out in meticolous detail in a PowerPoint before signing off on it and granting any type of access to data. Which means I need to create some really good looking PPT decks to even get the chance to explore our available data, it's quite frustrating and I'd rather spend the time doing something else.
By detail I mean like, budget, development timeline, target audience, documentation, blabla, before I even get a chance to look at the data and determine if it's useable in the first place.
Anyone else have this problem?
submitted by /u/NipponPanda
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
Usefulness of Six-Sigma
How useful would y'all rate a Six-Sigma certification?
submitted by /u/Slow_Act_4114
[link] [comments]
Usefulness of Six-Sigma
How useful would y'all rate a Six-Sigma certification?
submitted by /u/Slow_Act_4114
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
Data Science
Seeking Advice for Achieving a 0.97 Score
I'm currently working on a machine learning task on Kaggle, and I'm striving to achieve a minimum of 0.97 score in accuracy. While I've made some progress, I've hit a plateau at 0.91 and can't seem to improve beyond that.
Task Description: I'm working on a classification task where tweets need to be classified into two categories: "Sports" or "Politics." I've used various models, including BERT, and have explored hyperparameter tuning, but I haven't been able to achieve the desired accuracy.
Current State: My best model currently has an accuracy of 0.91. I'm looking for ideas, strategies, and any advice that might help me break through this barrier and achieve a 0.97 accuracy score. I'm open to trying new approaches or techniques, and I'd love to hear from anyone who has experience with similar tasks.
Questions:
* Are there specific techniques or approaches you recommend for improving model accuracy?
* How can I make the most out of my training data and optimize the model further?
* Any insights on feature engineering or data augmentation that could help?
I greatly appreciate any insights or feedback you can provide. Please share your experiences, suggestions, or any resources you think might be helpful.
submitted by /u/poolyhymnia
[link] [comments]
Seeking Advice for Achieving a 0.97 Score
I'm currently working on a machine learning task on Kaggle, and I'm striving to achieve a minimum of 0.97 score in accuracy. While I've made some progress, I've hit a plateau at 0.91 and can't seem to improve beyond that.
Task Description: I'm working on a classification task where tweets need to be classified into two categories: "Sports" or "Politics." I've used various models, including BERT, and have explored hyperparameter tuning, but I haven't been able to achieve the desired accuracy.
Current State: My best model currently has an accuracy of 0.91. I'm looking for ideas, strategies, and any advice that might help me break through this barrier and achieve a 0.97 accuracy score. I'm open to trying new approaches or techniques, and I'd love to hear from anyone who has experience with similar tasks.
Questions:
* Are there specific techniques or approaches you recommend for improving model accuracy?
* How can I make the most out of my training data and optimize the model further?
* Any insights on feature engineering or data augmentation that could help?
I greatly appreciate any insights or feedback you can provide. Please share your experiences, suggestions, or any resources you think might be helpful.
submitted by /u/poolyhymnia
[link] [comments]
Reddit
From the datascience community on Reddit
Explore this post and more from the datascience community
insideBIGDATA
Generative AI Highlights the Need for Identity Verification
https://insidebigdata.com/wp-content/uploads/2023/08/Generative_AI_shutterstock_2273007347_special.jpg In this contributed article, Mark Lieberwitz, Co-Founder & CPO of KarmaCheck, takes a look at how verified identities help restore trust in digital content, combat the erosion of trust caused by generative AI, and prevent the dissemination of false narratives. Implementing identity verification must be accompanied by robust privacy measures and industry-wide collaboration to protect user data and establish consistent standards.
Generative AI Highlights the Need for Identity Verification
https://insidebigdata.com/wp-content/uploads/2023/08/Generative_AI_shutterstock_2273007347_special.jpg In this contributed article, Mark Lieberwitz, Co-Founder & CPO of KarmaCheck, takes a look at how verified identities help restore trust in digital content, combat the erosion of trust caused by generative AI, and prevent the dissemination of false narratives. Implementing identity verification must be accompanied by robust privacy measures and industry-wide collaboration to protect user data and establish consistent standards.
insideBIGDATA
Generative AI Highlights the Need for Identity Verification
In this contributed article, Mark Lieberwitz, Co-Founder & CPO of KarmaCheck, takes a look at how verified identities help restore trust in digital [...]