Differences

This shows you the differences between two versions of the page.

--- task_boiler [2022/03/18 22:57] – raju
+++ task_boiler [2024/01/22 22:59] – [Issue] raju
@@ Line 4: / Line 4: @@
 Start the first heading with "====="
-===== convert a dictionary of dataframes to a big dataframe =====
+===== blocked by barrons =====
-==== Task ====
+==== Issue ====
-Given a dictionary where the keys are strings and values are dataframes, we want to create an expanded
+barrons.com is giving the following error.
-dataframe where the keys go into a column of their own. For example, given
 <code>
-{'COST':    fiscal_quarter_end  reported_date
+You have been blocked.
-              202102       20210304
-              202105       20210527
+Why?
-              202108       20210923
+Something about the behaviour of the browser has caught our
-              202111       20211209,
+attention.
- 'CPRT':    fiscal_quarter_end  reported_date
-              202104       20210519
+There are various possible explanations for this:
-              202107       20210908
+* You are browsing and clicking at a speed much faster than
-              202110       20211117
+  expected of a human being.
-              202201       20220216}
+* Something is preventing JavaScript from working on your
-</code>
+  computer.
-We want
+* There is a robot on the same network (IP WW.XX.YY.ZZ) as you.
-<code>
-  ticker  fiscal_quarter_end  reported_date
+Having problems accessing the site? Submit feedback.
-   COST              202102       20210304
+ID: a-b-c-d-e
-   COST              202105       20210527
-   COST              202108       20210923
-   COST              202111       20211209
-   CPRT              202104       20210519
-   CPRT              202107       20210908
-   CPRT              202110       20211117
-   CPRT              202201       20220216
 </code>
 ==== Solution ====
-<code>
+Click on the extensions button in the top right
-pd.concat(dict_df, axis=0).reset_index().drop('level_1', axis=1).rename({'level_0':'key_name'}, axis=1)
+-> Bypass Paywalls Clean -> settings button -> Options
-</code>
+-> scroll down to "BPC settings" section
-Using the example above
+-> check "Barron's - no Googlebot (http error 500)" option
-<code>
+-> Save
-$ ipython
-Python 3.6.12 |Anaconda, Inc.| (default, Sep  9 2020, 00:29:25) [MSC v.1916 64 bit (AMD64)]
-Type 'copyright', 'credits' or 'license' for more information
-IPython 7.16.1 -- An enhanced Interactive Python. Type '?' for help.
-In [1]:
+==== System Information ====
-import pandas as pd
+  * bypass paywalls clean extension version: 3.5.2.1 (latest asof 2024-01-22)
-costco_earnings = pd.DataFrame({
+  * google chrome: 120.0.6099.200
-  'fiscal_quarter_end': [202102, 202105, 202108, 202111],
+  * OS: Windows 10 Enterprise
-  'reported_date': [20210304, 20210527, 20210923, 20211209]
-})
-costco_earnings
-Out[1]:
-   fiscal_quarter_end  reported_date
-              202102       20210304
-              202105       20210527
-              202108       20210923
-              202111       20211209
-In [2]:
-copart_earnings = pd.DataFrame({
-  'fiscal_quarter_end': [202104, 202107, 202110, 202201],
-  'reported_date': [20210519, 20210908, 20211117, 20220216]
-})
-copart_earnings
-Out[2]:
-   fiscal_quarter_end  reported_date
-              202104       20210519
-              202107       20210908
-              202110       20211117
-              202201       20220216
-In [3]:
-dict_df = {'COST': costco_earnings, 'CPRT': copart_earnings}
-dict_df
-Out[3]:
-{'COST':    fiscal_quarter_end  reported_date
-              202102       20210304
-              202105       20210527
-              202108       20210923
-              202111       20211209,
- 'CPRT':    fiscal_quarter_end  reported_date
-              202104       20210519
-              202107       20210908
-              202110       20211117
-              202201       20220216}
-In [4]:
-expanded_df = pd.concat(dict_df, axis=0).reset_index().drop('level_1', axis=1).rename({'level_0':'ticker'}, axis=1)
-expanded_df
-Out[4]:
-  ticker  fiscal_quarter_end  reported_date
-   COST              202102       20210304
-   COST              202105       20210527
-   COST              202108       20210923
-   COST              202111       20211209
-   CPRT              202104       20210519
-   CPRT              202107       20210908
-   CPRT              202110       20211117
-   CPRT              202201       20220216
-</code>